Hi Martijn, Which compiler are you using?
On Sat, Jul 20, 2013 at 1:22 AM, Martijn Jasperse <[email protected]>wrote: > Hi all, > I'm writing data-logging software storing in an HDF5 file, to be run for > months at a time. I'm intending to use a chunked dataset, and append a row > every time a measurement is made by using H5Dwrite with a hyperslab > selection on the filespace. However, I've noticed that the memory used by > the application increases with every subsequent H5Dwrite call. > > I've simplified it down to a minimal example below. My understanding is > that since the dimensions are fixed at creation time there should be no > increase of memory during the loop. Any suggestions for why it should > continue to accumulate memory? Commenting out the line start[0]=i;results in > no memory increase. This example doesn't accumulate much memory > but my actual application runs out of memory and crashes after about a week. > > Am I doing something stupid? Is this expected behaviour? I'm running > HDF5 1.8.11 on Win7 32-bit. > > Cheers, > Martijn > > #include <hdf5.h> >> const int NITEMS = 10000; >> const int NREPS = 100000; >> int main() >> { >> int i; >> hid_t fp, sp, dset, memsp, props; >> float *arr = ( float* )calloc( NITEMS, sizeof( float )); >> hsize_t dims[] = { NREPS, NITEMS }; >> hsize_t start[] = { 0,0 }; >> hsize_t count[] = { 1,NITEMS }; >> /* invent data */ >> for ( i = 0; i < NITEMS; ++i ) arr[i] = i; >> fp = H5Fcreate( "test.h5",H5F_ACC_TRUNC,H5P_DEFAULT,H5P_DEFAULT ); >> if ( fp <= 0 ) return 1; >> /* create chunked dataset */ >> props = H5Pcreate( H5P_DATASET_CREATE ); >> H5Pset_chunk( props, 2, count ); >> sp = H5Screate_simple( 2,dims,NULL ); >> dset = H5Dcreate2( >> fp,"test",H5T_NATIVE_FLOAT,sp,H5P_DEFAULT,props,H5P_DEFAULT ); >> if ( dset <= 0 ) return 1; >> H5Pclose( props ); >> /* write row by row */ >> memsp = H5Screate_simple( 2,count,NULL ); >> for ( i = 0; i < NREPS; ++i ) >> { >> start[0] = i; >> H5Sselect_hyperslab( sp,H5S_SELECT_SET,start,NULL,count,NULL ); >> if ( H5Dwrite( dset,H5T_NATIVE_FLOAT,memsp,sp,H5P_DEFAULT,arr ) < 0 ) >> break; >> } >> H5Sclose( memsp ); >> H5Dclose( dset ); >> H5Fclose( fp ); >> } > > >
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
