--- Martin Kersten <[EMAIL PROTECTED]> wrote: > > > > During very large insert operations (total of > 100-250M > > rows broken into batches of 500K) the mserver5 > process > > will consume 100% of RAM and nearly 100% of swap. > I'm > > running on Linux 64bit with 9GB RAM and 2GB of > SWAP. > One cause of errors in this area is that people > forget > to announce the number of records that will be > copied. > This really makes a difference. > > This problem should be solved in the upcoming > release > (and CURRENT) since a few weeks. It now samples the > load file before BATs are allocated. > >
Indeed it made a big difference. From the documentation it seemed simply optional to leave off the number of records being copied. By adding it into my load script the memory exhaustion issue has been solved. MonetDB5 doesn't use swap now and performance is predictable again. I'm running a nightly build from the stable branch, BTW. ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
