Hello, I did a test here with a CSV file containing 50 000 records and it took 16 minutes.
I suspect this has to do with your Java configuration - have you e.g. set the environment variable JAVA_OPTS to allocate memory to Java ? A reasonable value would be -Xms500m -Xmx1000m -XX:PermSize=250m -XX:MaxPermSize=500m Lars On Sat, May 5, 2012 at 9:11 AM, Edward Ari Bichetero <ebich...@yahoo.com>wrote: > The import file contains 46800 records (individual CSV lines). > > > - Edward - > > ________________________________ > From: Lars Helge Øverland <larshe...@gmail.com> > To: Edward Ari Bichetero <ebich...@yahoo.com> > Cc: "dhis2-users@lists.launchpad.net" <dhis2-users@lists.launchpad.net> > Sent: Friday, May 4, 2012 6:07 PM > Subject: Re: [Dhis2-users] DHIS version 2.8 released > > > > > > On Fri, May 4, 2012 at 4:37 PM, Edward Ari Bichetero <ebich...@yahoo.com> > wrote: > > Congratulation on the new release, it is nice to see the new features > especially the CSV import functionality. > >However ... > > > > > >I've just had a go at importing a small set of historical data (Monthly > outpatient attendance by district) going back six years (2005-2011). This > is just two data elements (Outpatient attendance, Outpatient reattendance) > with four combo categories (Male, Female, Below 5 yrs old, 5 yrs and older) > for each of our 112 districts. The CSV data file is about 4.6 megabytes in > size. > > > >I gave up on watching the import process after an hour. At that point it > had been using 90% of our test servers memory (4GB) and burning 100% of one > cpu/core almost the entire time. This is just one of the smallest datasets > we would be looking to import. It appears that the CSV import in it's > current state is not able to cope with reasonably large data. Or am I > getting this wrong ? Do you have any ideas/workarounds ? > > > > > > Hi, approximately how many records are in your import file? > > Lars >
_______________________________________________ Mailing list: https://launchpad.net/~dhis2-users Post to : dhis2-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-users More help : https://help.launchpad.net/ListHelp
