Code checkpoint
Description:
The fractal heap code is reasonably stable and is passing all its
current tests, so checkpoint everything with CVS.
Also, add "statistics" module for v2 B-trees code, which is only a
stub right now.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
Linux 2.4 (chicago)
Bug fix
Description:
Clean up a few code mis-matches that crept in during the last checkin.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5commitest
New feature
Description:
Update "power of two" macro and add "general" log2() routine.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
New feature
Description:
Add "variable-length" 32-bit encode/decode macros, matching the other
64-bit ones, for later use in the fractal heap code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor for full h5committest
New features
Description:
Add "find node greater than or equal to key" and "remove first" operations
to skip lists.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Description: Made small changes to sec2 driver to reflect that VMS systems ino_t is 1dim array
of size 3; all changes are marked with H5_VMS variable.
Defined HDremove macro for VMS to remove multiple versions of the files.
Solution:
Platforms tested: heping, VMS server
Misc. update:
Description:
Two functions (HDsrand and HDrand) are not declared. They caused
compiling errors on Windows. Declare these two functions:
H5_DLL int HDrand();
H5_DLL void HDsrand(unsigned int seed);
Solution:
Platforms tested:
Windows XP and heping
Misc. update:
Bug fix/Feature
Description:
Modify the library to use rand_r, when available, instead of srand. If
rand_r is not available, it will try to use srandom/random, and then finally
fall back to srand/rand.
Solution:
A couple places in the library use the srand() function to seed the random
number generator. This can cause problems on certain platforms and could theoretically
cause problems for users who expect a certain sequence of random numbers following their
own call to srand().
Most platforms have an implementation of rand_r, which is identical to rand, except that
it allows for explicit storage of the seed value. The configure script will now check
for the existence of rand_r to facilitate its use in the library.
Two new functions are added, to replace the macros HDsrand/HDrand. These functions are HDrand and
HDsrand. HDrand will call rand_r, if available, or random(), if available, and fall back to rand() if neither
of those is available. HDsrand will store the seed value locally, if rand_r is available. Otherwise, it will call
srandom if available, and fall back to srand otherwise.
Platforms tested:
heping, mir. Really need to test on Red Storm, since that platform motivated this fix, but the machine
is currently not available.
Misc. update:
Bug fix
Description:
Correct problem with new log2(n) routine on 64-bit machines.
Solution:
Put some casts in to make certain that the values are computed as 32-bit
integers.
Platforms tested:
Linux 2.4 (mir) - 64-bit
New feature & code cleanup
Description:
Update fractal heap to be able to insert objects into a direct block
hanging off the header.
Extract "octal dump" code into separate routine so that both the local
heaps & fractal heaps can use it.
Other code cleanups & support to get this far.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Added HL examples
Description:
Examples existed for the HL lite, image, and table interfaces, but
were never included in the distro... until now!
Solution:
Added source files to CVS, updated Makefile and MANIFEST.
Platforms tested:
mir, heping, shanti
Porting.
Description:
Red Storm's lustre used srand and rand to initialize their user mode
of lustre. HDF5 also uses srand and that messes up Lustre.
Solution:
Use -DRED_STORM to block out the HDF5 code that calls srand.
This is a temporary fix since Lustre has fixed its code from
calling srand and rand but the fix is not deployed at Red_storm
yet.
Platforms tested:
Tested at Red-Storm.
Incrementtal checkin
Description:
Revise & update v2 B-tree code to separate the internal package-specific
routines from the library-callable "private" routines.
Similar updates for the fractal heap code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
New feature
Description:
Check in initial "fractal heap" code, for supporting the group redesign.
Also, remove some remnants of the segmented heap/block tracker/B+tree code
which I didn't find during the last pass.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
Linux 2.4
Maintenance
Description:
Remove half-built (half-baked? :-) implementation of B+tree, block tracker
and segmented heaps. (In preparation for starting the new heap coding)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
bug fix
Description:
ret_value is not set to SUCCEED in H5D_mpio_select_write
That causes the 64-bit intel compiler unhappy.
It will give a non-zero number and cause the testing fake failure.
Another one is picked up by cmpi again. For one optimization case,
another variable is not initialized properly and compiler set an unhappy number,
cause the test failed.
Solution:
Properly initialize those variables.
Platforms tested:
teragrid:parallel
mir:parallel
heping: parallel and sequential
tungsten:parallel
Misc. update:
h5committest doesn't finish due to no space left on device.
parallel tests still failed at tungsten with cmpi. It looks like it was a bug from cmpi.
bug fix for daily test
Description:
if (fm->select_chunk) is not NULL at sequential mode and not set by the library, it shouldn't be free.
Otherwise, it will cause failures in the daily test.
Solution:
Explicitly set (fm->select_chunk) to NULL for non-parallel case.
Platforms tested:
copper with parallel
heping with both sequential and parallel
shanti with sequential
Misc. update:
Hopefully only mir failed at daily test.
Bug fix in collective IO code
Description:
When select none occurs in one process, fm->select_chunk hasn't been allocated the memory(not allocated until create_file_hyper); however, this needs to be checked; it was caught with cmpi at NCSA linux cluster.
Solution:
Relocate the place to allocate the memory space for fm->select_chunk.
Platforms tested:
cmpi at tungsten
mpich at heping
kind of minor and let daily test catch other errors.
Misc. update:
This won't fix the failure at mir.
bug fix
Description:
During the process of the combination of collective chunk IO code and the original IO code,
The return value of a function call hasn't been initialized. On 32-bit mode at modi4, that value isn't set to 0 by default. So a fake error value returns.
Solution:
Re-arrange the code so that the return value will be set properly.
Platforms tested:
parallel:
modi4 - 32bit
heping
seq:
heping
Misc. update:
Enhanced collective chunk IO supports
Description:
1. When using collective IO with chunking storage without any tuning, performance may become worse under some circumstances.
2. Current HDF5 handles raw-data IO per chunk. So for many small chunks, many small IOs will be passed into MPI-IO. That may cause bad performance.
3. For one IO per chunk case, sometimes performance with collective is worse than performance with independent. An obvious case is when only one process is doing IO and all other process are not doing IO, the collective IO will only add overheads for communication. We want to avoid this case. Some management inside our library needs to be done.
Solution:
- Added managements of collective IO supports for chunking storage
inside parallel HDF5
1) Implemented One IO with collective mode for all chunks in the
application by building one MPI derived datatype accross all
chunks.
2) Implemented the decision-making support to do collective IO inside
MPI-IO per chunk.
3) Added the decision-making support to do one IO accross all chunks
or to do multiple IOs with each IO per chunk.
4) Added the support to handle the case some processes won't do any IOs in
collectively.
5) Some MPI-IO package(mpich 1.2.6 or lower, e.g.) cannot handle
collective IO correctly for the case when some processes have no
contributions to IOs, a special macro is added to change
collective IO mode to independent IO mode inside HDF5 library.
Platforms tested:
Parallel:
IBM AIX 5.2(copper)
Linux (heping) mpich-1.2.6
SDSC Teragrid mpich-1.2.5
Linux(Tungsten) mpich-1.2.6
Altix(NCSA cobalt)
Seq:
Linux(heping)
Misc. update:
Bug fix & new feature
Description:
Support variable-length datatypes in compact data storage and chunked
data storage, along with attributes.
Bug fix on the H5T_vlen_set_loc to allow for changing the file on a
variable-length datatype on disk.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Can't h5committest right now, due to missing cache files.
1) Add parallel test for metadata cache
2) Split serial test for metadata cache into two parts
3) Fix bug in which cache was flushed needlessly when the
cache wasn't full.
4) Performance improvements
5) Update API for parallel cache coherency bug fix.
Description:
See above.
Solution:
See above.
Platforms tested:
h5committest
Misc. update:
Code cleanup
Description:
Check in some of the code cleanups from working on the external link
support. (This doesn't include any of the external link features)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX.4 (amazon)
Linux 2.4
Cleaned up reconfigure output
Description:
Added two lines to separate output from trace script from autotools
output in reconfigure.
Solution:
Running reconfigure updated src/Makefile.in as a side effect, since
I changed config/lt_vers.am in my last checkin but forgot to
run reconfigure then.
Platforms tested:
mir (cosmetic change to reconfigure script only)
Code cleanup/update
Description:
Update compiler flags for gcc 4.1.x & start cleaning up some of the C++
compatibility
Platforms tested:
FreeBSD 4.11 (sleipnir) w/gcc 4.1 and/or g++
Feature
Description:
Added character encoding and attribute creation property lists.
Solution:
Attributes' character encoding is set via the ACPL.
The default is ASCII, with UTF-8 being the other option currently.
Platforms tested:
heping, mir, sleipnir, copper
Bug fix
Description:
Retrieving an object's name could fail (in various ways) under certain
circumstances (mostly having to do with mounted files).
Solution:
Re-write & simplify "get object name" code to fix error in a better way
than adding yet another hack to the previous pile of hacks... :-)
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Patch for red-storm.
Description:
A temporary patch for redstorm, must open a file in the same parallel
data filesystem before H5_init_library. Otherwise, it hangs with error:
(client.c:568:ptlrpc_check_status()) @@@ type == PTL_RPC_MSG_ERR, err == -114 req@0x57a9660 x1/t0 o38->mds_l2@MDS_PEER_UUID:12 lens 240/272 ref 1 fl Rpc:R/100000/0 rc 0/-114
Platforms tested:
Red-storm.
Code cleanup
Description:
Fix a bunch of warnings flagged by Windows compilers.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Unstable format version marker.
Description:
This marks the Check in baseline for compact group revisions,
which radically revises the source code for managing groups and object headers.
Earlier versions (1.7.52 or earlier) have the original format. This version
and later have the unstable format until further notice.
Platforms tested:
h5committested.
New feature
Description:
Add in a combination of Peter's & my code to support copying
variable-length data from one file to another, although currently only
supported with contiguous data storage.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Description: H5Eset_auto_stack() and H5Eset_auto() used FUNC_ENTER_API. It would clear the error
stack from the previous API call. It's better to use FUNC_ENTER_API_NOCLEAR to avoid clearing
the stack.
Platforms tested: fuss - simple change.
Bug fix
Description:
Fortran type generation was broken in two ways. Fixed both.
Solution:
Firstly, there were a couple of path problems. Fixed a typo and
specified the full path of a file.
Secondly, the dependencies weren't right when building with HDF5-specific
commands (make lib, make check-s, etc.). Tweaked dependencies
to fix the problem.
Platforms tested:
mir, modi4, sleipnir
slight code change for efficiency of iteration of chunk address
Description:
Quincey picked this up because of my inexperience of C. A pair of redundant malloc and free calls will be removed.
Solution:
Platforms tested:
heping(Linux 2.4)
too little to be tested.
Misc. update:
Quincey Koziol
HDF Admin
Elena Pourmal
Fang Guo
Leon Arber
James Laird
Albert Cheng
MuQun Yang
John Mainzer
Raymond Lu