Code cleanup
Description:
Add & improve assertion that was removed in last commit, to check for
compact dataset storage.
Platforms tested:
FreeBSD 4.8 (sleipnir)
triple check not necessary.
Silly bug fix.
Description:
Forgot to put the comm/info duplication code in.
Solution:
Put them back in.
Platforms tested:
Did not run h5committest since the changes are all in the MPIO driver code.
Tested on eirene and Modi4, both parallel modes.
Misc. update:
Commenting
Description:
Added comments to the H5Fget_access_plist function to remind people
that if they are going to be overwriting a value in the plist which
was originally opened and inserted into the plist, then it should be
closed before overwriting that value.
Platforms tested:
Modi4 (Only comment change, so no h5committest needed).
Misc. update:
bug fix
Description:
in the close strong case in H5F_close there was a do..while loop
that executed when nfiles==0, calling malloc(0)
this was causing a failure in the close call in Code Warrior, that returns 0
in the call pt=malloc(0)
Solution:
replaced with a while loop (checks the condition before executing)
Platforms tested:
code warrior, linux
Misc. update:
Bug Fix
Description:
A resource leak happened if the H5Fget_access_plist() function was
called. What was happening: the driver ID and info parts of the
property list copied in H5Fget_access_plist were being overwritten,
but those properties were copied initially, so we lost information.
Solution:
Before copying over those values, call the H5F_acs_close() function
to close those values.
Platforms tested:
Modi4 (Parallel & Fortran)
Arabica (Fortran)
Verbena (Fortran & C++)
Misc. update:
Bug fix
Description:
Sometimes when H5detect fails (incorrect code, incorrect mpi launch
command, insufficient resources,...), it generates an incomplete or
even empty H5Tinit.c file. If the empty file is not removed but
the make is run again, make will just use the empty file to generate
the library which will have lots of missing routines. Make won't
fail until much later and the missing routines would be puzzling
to inexperienced users.
Solution:
If H5detect fails, the generated H5Tinit.c will be removed unless
$HDF5_Make_Ignore is set. This is similar to removing .o file if
the compile fails.
Platforms tested:
Copper (parallel) which has a condition causing H5detect to fail.
That verified the change works.
Eirene (parallel) in which H5detect runs well. This verified the
change works in normal conditions too.
Did not do the h5committest because this is just a makefile change
and the above two tests covered it well.
Misc. update:
Bug fixes/API changes
Description:
Previously, the Communicator and Info object arguments supplied
to H5Pset_fapl_mpio() are stored in the property with its handle
values. This meant changes to the communicator or the Info object
after calling H5Pset_fapl_mpio would affect the how the property
list function. This was also the case when H5Fopen/create operated.
They just stored the handle value. This is not according to the
MPI-2 defined behavior of how Info objects should be handled. (MPI-2
defines Info objects must be parsed when called.)
The old design was trying to avoid numerous duplicates of the same
information (e.g., every property object holds one version, every
file opened holds another version, when all of them are referring to
the same original version.) Nevertheless it is safer to implement
it according to MPI-2 definition.
Futhermore, the library often needs to do message passing using the
supplied communicator. Using the same communicator as the application
version may result in some messages mix up.
Solution:
H5Pset_fapl_mpio now stores a duplicate of each of the communicator
and Info object.
H5Pget_fapl_mpio returns a duplicate of its stored communicator and
Info object. It is now the responsibility of the applications to free
those objects when done.
H5Fopen/create also stores a duplicate of the communicator and Info
object supplied by the File Access Property list.
H5Fclose frees those duplicates.
There are a few more internal VFL call back functions that they
follow this "make duplicates" requirement.
Platforms tested:
"h5committested".
What other platforms/configurations were tested?
Eirene (mpicc), sol(mpicc), copper(parallel)
Misc. update:
A compiling error fixed for windows only
Description:
Use file handler instead of file stream at H5FDstdio.c when using getfileHandle function of windows.
Solution:
see above, however; stdio driver tests failed on windows. I am still investigating the bug.
The current check will not fix the failure of stdio driver test on windows.
The good news is that it won't affect the release since the default driver used for test is sec2 driver and
all tests passed for sec2 driver on windows.
Platforms tested:
windows 2000 and confirmed at Linux.
Since the only change is two-line code inside #ifdef WIN32 #endif macro block,
it is not necessary to test all UNIX platforms. Still confirmed at eirene.
Misc. update:
Bug fix.
Description:
The stdio filer driver is not reducing the file's size in the manner
that the sec2 driver does.
Solution:
Copy code from the sec2 for handling this properly.
Platforms tested:
h5committested
Bug Fix
Description:
When calling "H5F_get_access_plist" after setting the FAPL to a multi
driver, and then trying to get the multi driver back, it resulted in
returing a "NULL" value instead of the driver.
Solution:
The stuff at the location pointed to by the driver was being stored
into the property list and not the pointer to the driver itself.
Changed the "H5P_set" code so that it stores the pointer instead of
the driver...(Added an "&" in from of the "driver_info" variable).
Platforms tested:
Arabica (Fortran)
Burrwhite (Fortran & C++)
Modi4 (Fortran & Parallel)
Misc. update:
New feature, sorta
Description:
Track changes to various internal APIs
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
New feature
Description:
Track changes in the internal metadata API with clearing the metadata
dirty flag without flushing object.
Added ability to delete an object header in the file and restore all the
space referenced by various header messages.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Code cleanup, bug fix & new feature.
Description:
The local heaps were missed in the last changes to the metadata "dirty"
flag, making them operate incorrectly in the metadata cache. Fixed by changing
to use "cache_info.dirty" flag instead.
Track changes for internal metadata API, regarding marking metadata as clean
without flushing it to the file.
Added ability to delete a local heap in the file.
Added ability to reduce the size of a local heap if there is a free-space
block at the end of the heap and the free-space block is over half the size of
the local heap.
Some migration of package-specific information out of the private header
and into the code module as well.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Bug fix & new feature.
Description:
The global heap 'dirty' flag was missed in the last round of changes to
the metadata cache and was not using the new "cache_info.dirty" flag.
Solution:
Switched to using correct flag for marking dirty objects in the metadata
cache.
Also, added new metadata callback for marking a piece of metadata as clean
without writing it back to the file.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
New feature.
Description:
Added ability to release space used for storing symbol table entries and
nodes when a group is deleted in a file.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Code cleanup
Description:
Stop using hard-coded constant for the initial size of the root group's
symbol table local heap and use macro.
Also, changed existing (unused) macro for the initial heap size to be the
same as the hard-coded constant.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
New feature.
Description:
Added ability to free space used for storing raw data for both contiguous
and chunked datasets.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
New feature.
Description:
Update prototypes for recent internal API addition.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
New feature.
Description:
Mark an object in the "open object list" as deleted when it's reference
count within the file is reduced to zero and it is only being held open by an
ID in the user's application.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
New feature.
Description:
Allow the file to shrink as well as expand when flushing a file before
closing it.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Bug fix.
Description:
Corrected fencepost error which was incorrectly checking for merging
metadata at the front of the metadata accumulator. This now allows the
metadata cache to cache I/O operations more efficiently.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
New code.
Description:
Added internal routine to clear the sieve buffer if it overlaps a address-
length range in the file. This is used currently to clear the sieve buffer
when an object is deleted from the file.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Code cleanup & new feature
Description:
Track changes to H5AC API (code cleanup)
Added routine to delete B-tree in file.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Code cleanup, bug fix & new feature.
Description:
Moved some "package-scope" information out of the private header file and
into the code module. (code cleanup).
Fixed a few places where the "destroy but don't flush" code wasn't working
correctly, which mostly affects the FPH5 effort. (bug fix).
Added the ability to "unprotect" a metadata object and delete it at the
same time. (new feature).
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Code cleanup
Description:
Uncompressed buffers can't get to the szip filter's decompression code now
that they are handled correctly by the chunk's filter mask.
Solution:
Remove handling of uncompressed buffers from szip filter's decompression
code.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
h5committest not necessary & doesn't test szip code.
Fix
Description:
This is an analogue to the previous bug-fix for filters not being
applied to data written but being applied when read. The old way was
if the SZlib library couldn't deflate a dataset, then we'd munge
along pretending that it was okay.
Solution:
Trigger it as an error in this situation. The H5Z_pipeline code which
calls this function can clear the error stack if need be.
Platforms tested:
Modi4 (Parallel & Fortran)
Burrwhite (Fortran & C++)
Baldric (Fortran), but make check didn't work because of "libucb.so"
error that I can't fix...)
Misc. update:
Bug fix
Description:
This fixes a bug in the low-level metadata caching code in the library
which could possibly lose metadata during file I/O when a lot of objects are
inserted into a group.
This also fixes a couple of (similar) fencepost bugs in the B-tree
deletion code.
Solution:
For the metadata bug - call the low-level driver's 'write' routine instead
of H5FD_write.
For the B-tree bug - include the correct number of keys.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
(h5committest is still not working for me on burrwhite)
Misc. update:
Buglet Fix
Description:
When filtering data, those filters which failed and are optional
weren't being specified inside of the H5Z_pipeline() function like
they're supposed to be.
Solution:
Modified the code to set the appropriate bit in the "failed" flag to
say that the filter failed if it did and was optional. Removed some
cases of setting the "failed" flag where it didn't have any effect
(like, right before an HGOTO_ERROR() statement).
Platforms tested:
Burrwhite (Fortran)
Arabica (Fortran)
Modi4 (Parallel & Fortran)
Misc. update:
Code cleanup/new features
Description:
Switch over to a new style for registering filters with the library -
instead passing in an ID, a string and a callback function to H5Zregister,
the client should pass in a single pointer to a H5Z_claass_t struct which
contains the ID, the description string and all the function callbacks as
fields.
Added support for a new "can apply" callback for each filter, which is
called when a dataset is created to check whether the parameters for that
filter apply correctly to the combination of the datatype and the chunk size
(i.e. dataspace) for the dataset.
Added support for a new "set local" callback for each filter, which
is called when a dataset is created (after the "can apply" filter callback)
and sets filter parameters that are specific to that particular dataset.
Switched the filters we ship over to use the new H5Z_class_t struct for
their internal registrations and also added "set local" callbacks to the
szip and shuffle filters and a "can apply" callback to the szip filter.
Lots of other code cleanups, etc. also
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
Linux 2.4 (sleipnir) w/szip
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel
Misc. update:
Code cleanup & new features
Description:
Added three new API functions for working on I/O filters: H5Pmodify_filter,
H5Pget_filter_by_id and H5Pall_filters_avail. Documentation on these will be
added for the 1.6.0 release, until then, we'll have to use the source code.
Also, simplified the H5Pset_szip and H5Pset_shuffle API interfaces, since
those filters are using the new "set local" callback facility to set dataset-
specific parameters.
Probably, some of the changes to internal H5Z routines are captured in
these changes also.
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
Linux 2.4 (sleipnir) w/szip
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel
Misc. update:
Code cleanup
Description:
Got rid of anonymous inline struct for filter information in a pipeline
and replaced with typedef'd type.
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
Linux 2.4 (sleipnir) w/szip
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel
Misc. update:
Code cleanup
Description:
Array declaration was using hard-coded constant for maximum number of
dimensions.
Solution:
Changed to use H5O_LAYOUT_NDIMS.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
Linux 2.4 (sleipnir) w/szip
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel
Misc. update:
New feature
Description:
Hooked up internal dataset creation call (H5D_create) to the new "can apply"
and "set local" callbacks for filters.
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
Linux 2.4 (sleipnir) w/szip
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel
Misc. update:
code warrior port
Description:
added some type casts (char*) for CW portability
Solution:
Platforms tested:
w2000(CW , MS)
linux
arabica
modi4
Misc. update:
Code cleanup & bug fix
Description:
Cleaned up comments & copyright statement.
Blocks which were unable to be compressed were missing the "length" prefix,
causing the decompression routine to fail.
Solution:
Always add "length" prefix to all blocks, even ones the fail to compress.
Platforms tested:
FreeBSD 4.8 (sleipnir)
IRIX64 6.5 (modi4) w/parallel
Misc. update:
Code cleanup
Description:
Added lots of comments to existing filters, so they are easier to understand
and use as examples for future filters. Cleaned up various bits of code, etc.
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir)
IRIX64 6.5 (modi4) w/parallel
Misc. update:
Code cleanup (sorta)
Description:
The H5Pset_szip API call was forcing users to pass in their parameters
in the same way as the internal representation of the filter parameters used
by the library (as an array of unsigned values).
Solution:
Changed to pass in separate parameters in a more user-friendly format.
Platforms tested:
FreeBSD 4.8 (sleipnir)
IRIX64 6.5 (modi4) w/parallel
Misc. update:
Bug Fix
Description:
Was using the wrong pointer into a structure...
Solution:
Used the correct pointer to get the file_id...
Platforms tested:
Modi4
Misc. update:
Quincey Koziol
Raymond Lu
Albert Cheng
Bill Wendling
Pedro Vicente Nunes
MuQun Yang
HDF Admin