mirrors/hdf5
2
0
Fork 0
mirror of https://github.com/HDFGroup/hdf5.git synced 2026-09-25 04:09:44 +03:00
Code Issues Packages Projects Releases Wiki Activity
Files
develop
hdf5/java/src-jni/test/TestH5Arw.java
T
Matt Landgithub-actions b58ab3cc1f Fix bad reads of nested cmpd/vlen types in JNI (#6413) 
* Fix H5DreadVL failing for pre-allocate cmpd-of-seq dsets

* Fix bad vlen of cmpd with null slot read

* Fix bad cmpd of cmpd read in java

`translate_rbuf`'s H5T_VLEN case had a similar bug where when `found_jList` was set to false due to an entyr in `ret_buf` being null, `ret_buf.add()` would be invoked on an array of objects without the list .add() method. This would occur whenever a read was invoked of a vlen sequence with a null (non-preallocated) entry. The pre-existing tests only tested the pre-allocated cases.

I removed the use of the `found_jList` flag, since it conflated the passing of an unallocated slot with `ret_buf` not being an array. Instead use `ret_buflen == 0` as the check to match the pattern in H5T_INTEGER and other branches.

The test for this fix is testH5Dread_vlen_of_compound_nullslot.

---

`translate_atomic_rebuf` had two issues related to handling of nested compounds. First, it discarded recursive returns, resulting in the construction of empty lists. Secondly, its member offset (`char_buf + i * typeSize + memb_offset`) was incorrect. In this case, `i` was the member index and `memberSize` was the entire cmpd size, so the offset would be erroneously large. It seems like this came from copying of the offset computation from `translate_rbuf`, which had to advance over entire  elements of compound data. This error was duplicated on the write side in `translate_atomic_wbuf`'s H5T_COMPOUND case (h5util.c:4611).

I changed `translate_atomic_rbuf` to capture the resultant object, and dropped the `i * typeSize` term in both routines.
The new test verifying the fix works is `testH5Dread_vlen_of_nested_compound`.

* Add exception checks

* Update NULL checks in translate_wbuf

* Correct potentially bad array length check

* Clang format

* Fix readVL/writeVL crash on malformed buffer

* Committing clang-format changes

* Add bufSize checks to wbuf/rbuf translation

* Remove vlen pre-allocation support

* Harden JNI buffer interface

* Handle opaque types as byte[] and document JNI buffer data model

Opaque elements were grouped with H5T_INTEGER in the nested-type
translation path, which boxed them as Integer/Long and rejected
arbitrary-sized opaque blobs. Treat H5T_OPAQUE like H5T_REFERENCE
(a byte[] per element) in translate_atomic_rbuf, translate_atomic_wbuf,
and h5validate_atomic_wbuf so nested opaque round-trips correctly.

Also add "Buffer data model" header comments on translate_rbuf() and
translate_wbuf() and note the reference/opaque byte[] leaves in the
H5.java javadocv.

* Initialize typeSize to fix -Werror=maybe-uninitialized

typeSize was assigned only inside the vl_data_class branch but read in
a second, separate vl_data_class branch, which gcc -O2 flags as
maybe-uninitialized under -Werror. Initialize it to 0 at declaration in
H5Aread/H5Awrite/H5Dread/H5Dwrite, matching the existing vl_array_len
pattern.

* Port nested cmpd/vlen tests to java/test and sync reference

The legacy java/test tree's JUnit-TestH5D.txt reference listed the new
nested compound/vlen tests, but the corresponding @Test methods existed
only in java/src-jni/test/TestH5D.java. Port the 10 tests and the
writeCompoundOfVlenDataset helper into java/test/TestH5D.java, remove
debug prints, and
regenerate the reference to match the actual JUnit output.

* Support nested vlen/compound datatypes in Java FFM compat layer

The FFM compatibility layer (java/hdf) lacked the vlen/compound read and
write support that the JNI interface gained, so the nested cmpd/vlen tests
ported into java/test (TestH5D) failed and leaked an id.

VLDataConverter now has recursive encodeValue/decodeValue helpers that pack
and unpack any member class (integer, float, fixed/vl string, nested
compound, and VLEN) in the native HDF5 in-memory layout. These are wired
into convertCompoundDatatype, readCompoundDatatype and convertRawDataToArrayList,
and a type-aware convertToHVLAuto handles top-level VLEN-of-compound writes.
Compound reads now reclaim VL memory, and type/count mismatches raise
IllegalArgumentException instead of silently corrupting data.

H5DwriteVL rejects an undersized buffer up front and routes VLEN writes
through convertToHVLAuto. The JUnit-TestH5D reference regains its trailing
blank line to match the actual JUnit output.

* Committing clang-format changes

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-06 09:52:29 -05:00

483 lines
14 KiB
Java
Raw Permalink Blame History

/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the LICENSE file, which can be found at the root of the source code *
* distribution tree, or in https://www.hdfgroup.org/licenses. *
* If you do not have access to either file, you may request a copy from *
* help@hdfgroup.org. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
package test;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
import java.io.File;
import hdf.hdf5lib.H5;
import hdf.hdf5lib.HDF5Constants;
import hdf.hdf5lib.HDFNativeData;
import hdf.hdf5lib.callbacks.H5A_iterate_cb;
import hdf.hdf5lib.callbacks.H5A_iterate_t;
import hdf.hdf5lib.exceptions.HDF5Exception;
import hdf.hdf5lib.exceptions.HDF5LibraryException;
import org.junit.After;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestName;
public class TestH5Arw {
@Rule
public TestName testname = new TestName();
private static final String H5_INTS_FILE = "tintsattrs.h5";
private static final String H5_FLTS_FILE = "tfloatsattrs.h5";
private static final int DIM_X = 8;
private static final int DIM8_Y = 8;
private static final int DIM16_Y = 16;
private static final int DIM32_Y = 32;
private static final int DIM64_Y = 64;
private static final int DIM128_Y = 128;
private static final String DATASETU08 = "DU08BITS";
private static final String DATASETS08 = "DS08BITS";
private static final String DATASETU16 = "DU16BITS";
private static final String DATASETS16 = "DS16BITS";
private static final String DATASETU32 = "DU32BITS";
private static final String DATASETS32 = "DS32BITS";
private static final String DATASETU64 = "DU64BITS";
private static final String DATASETS64 = "DS64BITS";
private static final String DATASETF32 = "DS32BITS";
private static final String DATASETF64 = "DS64BITS";
private static final String DATASETF128 = "DS128BITS";
private static final int RANK = 2;
long H5fid = HDF5Constants.H5I_INVALID_HID;
long H5aid = HDF5Constants.H5I_INVALID_HID;
long H5did = HDF5Constants.H5I_INVALID_HID;
private final void _closeH5file() throws HDF5LibraryException
{
if (H5aid >= 0)
try {
H5.H5Aclose(H5aid);
}
catch (Exception ex) {
}
if (H5did >= 0)
try {
H5.H5Dclose(H5did);
}
catch (Exception ex) {
}
if (H5fid > 0)
try {
H5.H5Fclose(H5fid);
}
catch (Exception ex) {
}
}
public void openH5file(String filename, String dsetname)
{
try {
H5fid = H5.H5Fopen(filename, HDF5Constants.H5F_ACC_RDONLY, HDF5Constants.H5P_DEFAULT);
}
catch (Throwable err) {
err.printStackTrace();
fail("TestH5Arw._openH5file: " + err);
}
assertTrue("TestH5Arw._openH5file: H5.H5Fopen: ", H5fid >= 0);
try {
H5did = H5.H5Dopen(H5fid, dsetname, HDF5Constants.H5P_DEFAULT);
}
catch (Throwable err) {
err.printStackTrace();
fail("TestH5Arw._openH5file: " + err);
}
assertTrue("TestH5Arw._openH5file: H5.H5Dopen: ", H5did >= 0);
try {
H5aid = H5.H5Aopen(H5did, dsetname, HDF5Constants.H5P_DEFAULT);
}
catch (Throwable err) {
err.printStackTrace();
fail("TestH5Arw._openH5file: " + err);
}
assertTrue("TestH5Arw._openH5file: H5.H5Aopen: ", H5aid >= 0);
}
@After
public void closeH5file() throws HDF5LibraryException
{
if (H5aid >= 0)
try {
H5.H5Aclose(H5aid);
}
catch (Exception ex) {
}
if (H5did >= 0)
try {
H5.H5Dclose(H5did);
}
catch (Exception ex) {
}
if (H5fid > 0)
try {
H5.H5Fclose(H5fid);
}
catch (Exception ex) {
}
H5fid = HDF5Constants.H5I_INVALID_HID;
H5did = HDF5Constants.H5I_INVALID_HID;
H5aid = HDF5Constants.H5I_INVALID_HID;
System.out.println();
}
@Before
public void verifyCount() throws NullPointerException, HDF5Exception
{
assertTrue("H5 open ids is 0", H5.getOpenIDCount() == 0);
System.out.print(testname.getMethodName());
}
@Test
public void testH5Aread_8bit_ints()
{
byte[][] attr_data = new byte[DIM_X][DIM8_Y];
try {
openH5file(H5_INTS_FILE, DATASETU08);
}
catch (Throwable err) {
err.printStackTrace();
fail("testH5Aread_8bit_ints: openH5file: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_UINT8, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_8bit_ints: H5Aread: " + err);
}
// End access to the attribute and release resources used by it.
try {
H5.H5Aclose(H5aid);
}
catch (Exception err) {
err.printStackTrace();
}
// End access to the dataset and release resources used by it.
try {
H5.H5Dclose(H5did);
}
catch (Exception err) {
err.printStackTrace();
}
// Open an existing dataset.
try {
H5did = H5.H5Dopen(H5fid, DATASETS08, HDF5Constants.H5P_DEFAULT);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_8bit_ints: H5Dopen: " + err);
}
// Open an existing attribute.
try {
H5aid = H5.H5Aopen(H5did, DATASETS08, HDF5Constants.H5P_DEFAULT);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_8bit_ints: H5Aopen: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_INT8, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_8bit_ints: H5Aread: " + err);
}
}
@Test
public void testH5Aread_16bit_ints()
{
short[][] attr_data = new short[DIM_X][DIM16_Y];
try {
openH5file(H5_INTS_FILE, DATASETU16);
}
catch (Throwable err) {
err.printStackTrace();
fail("testH5Aread_16bit_ints: openH5file: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_UINT16, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_16bit_ints: H5Aread: " + err);
}
// End access to the attribute and release resources used by it.
try {
H5.H5Aclose(H5aid);
}
catch (Exception err) {
err.printStackTrace();
}
// End access to the dataset and release resources used by it.
try {
H5.H5Dclose(H5did);
}
catch (Exception err) {
err.printStackTrace();
}
// Open an existing dataset.
try {
H5did = H5.H5Dopen(H5fid, DATASETS16, HDF5Constants.H5P_DEFAULT);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_16bit_ints: H5Dopen: " + err);
}
// Open an existing attribute.
try {
H5aid = H5.H5Aopen(H5did, DATASETS16, HDF5Constants.H5P_DEFAULT);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_16bit_ints: H5Aopen: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_INT16, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_16bit_ints: H5Aread: " + err);
}
}
@Test
public void testH5Aread_32bit_ints()
{
int[][] attr_data = new int[DIM_X][DIM32_Y];
try {
openH5file(H5_INTS_FILE, DATASETU32);
}
catch (Throwable err) {
err.printStackTrace();
fail("testH5Aread_32bit_ints: openH5file: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_UINT32, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_32bit_ints: H5Aread: " + err);
}
// End access to the attribute and release resources used by it.
try {
H5.H5Aclose(H5aid);
}
catch (Exception err) {
err.printStackTrace();
}
// End access to the dataset and release resources used by it.
try {
H5.H5Dclose(H5did);
}
catch (Exception err) {
err.printStackTrace();
}
// Open an existing dataset.
try {
H5did = H5.H5Dopen(H5fid, DATASETS32, HDF5Constants.H5P_DEFAULT);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_32bit_ints: H5Dopen: " + err);
}
// Open an existing attribute.
try {
H5aid = H5.H5Aopen(H5did, DATASETS32, HDF5Constants.H5P_DEFAULT);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_32bit_ints: H5Aopen: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_INT32, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_32bit_ints: H5Aread: " + err);
}
}
@Test
public void testH5Aread_64bit_ints()
{
long[][] attr_data = new long[DIM_X][DIM64_Y];
try {
openH5file(H5_INTS_FILE, DATASETU64);
}
catch (Throwable err) {
err.printStackTrace();
fail("testH5Aread_64bit_ints: openH5file: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_UINT64, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_64bit_ints: H5Aread: " + err);
}
// End access to the attribute and release resources used by it.
try {
H5.H5Aclose(H5aid);
}
catch (Exception err) {
err.printStackTrace();
}
// End access to the dataset and release resources used by it.
try {
H5.H5Dclose(H5did);
}
catch (Exception err) {
err.printStackTrace();
}
// Open an existing dataset.
try {
H5did = H5.H5Dopen(H5fid, DATASETS64, HDF5Constants.H5P_DEFAULT);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_64bit_ints: H5Dopen: " + err);
}
// Open an existing attribute.
try {
H5aid = H5.H5Aopen(H5did, DATASETS64, HDF5Constants.H5P_DEFAULT);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_64bit_ints: H5Aopen: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_INT64, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_64bit_ints: H5Aread: " + err);
}
}
@Test
public void testH5Aread_32bit_floats()
{
float[][] attr_data = new float[DIM_X][DIM32_Y];
try {
openH5file(H5_FLTS_FILE, DATASETF32);
}
catch (Throwable err) {
err.printStackTrace();
fail("testH5Aread_32bit_floats: openH5file: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_FLOAT, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_32bit_floats: H5Aread: " + err);
}
for (int i = 0; i < DIM_X; i++)
assertTrue("testH5Aread_32bit_floats - H5.H5Aread: ", attr_data[i][0] == (32 - i));
}
@Test
public void testH5Aread_64bit_floats()
{
double[][] attr_data = new double[DIM_X][DIM64_Y];
try {
openH5file(H5_FLTS_FILE, DATASETF64);
}
catch (Throwable err) {
err.printStackTrace();
fail("testH5Aread_64bit_floats: openH5file: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_DOUBLE, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_64bit_floats: H5Aread: " + err);
}
for (int i = 0; i < DIM_X; i++)
assertTrue("testH5Aread_64bit_floats - H5.H5Aread: ", attr_data[i][0] == (64 - i));
}
@Test
public void testH5Aread_128bit_floats()
{
byte[][][] attr_data = new byte[DIM_X][DIM128_Y][16];
try {
openH5file(H5_FLTS_FILE, DATASETF128);
}
catch (Throwable err) {
err.printStackTrace();
fail("testH5Aread_128bit_floats: openH5file: " + err);
}
// Read data.
try {
H5.H5Aread(H5aid, HDF5Constants.H5T_NATIVE_LDOUBLE, attr_data);
}
catch (Exception err) {
err.printStackTrace();
fail("testH5Aread_128bit_floats: H5Aread: " + err);
}
}
}
Reference in New Issue View Git Blame Copy Permalink