Re: [Dhis2-users] Mydatamart issue?

2012-05-07 Thread Bob Jolliffe
Sorry not as quick as Ola and Lars .. yes do try those approaches first. We can also look at creating custom views in mydatamart if you want to filter the dataelements/indicators further. On 7 May 2012 19:02, Muhire Andrew wrote: > Thanks all for the quick supportive advices,i think if we do 1)

Re: [Dhis2-users] Mydatamart issue?

2012-05-07 Thread Muhire Andrew
Thanks all for the quick supportive advices,i think if we do 1) aggregation level District and also removing zero values will help. Dear Ola please publish this section , it will be helpfull to us. Thanks. Muhire Andrew HMIS/Ministry of Health First say

Re: [Dhis2-users] DHIS version 2.8 released

2012-05-07 Thread Lars Helge Øverland
Hi again. Thanks for making us aware of this. I have now corrected a flaw which caused slow handling of periods for each import record. After applying the fix my test with 50 k records are down to 20 sec. Note that I cleared out existing records from my database - new records are faster since we

Re: [Dhis2-users] DHIS version 2.8 released

2012-05-07 Thread Edward Ari Bichetero
Thanks for the reply. I'll check our Java setup, retry the import and report back. However there is still a larger issue. Taking 16 minutes as a representative timing and assuming linear scaling (unlikely but for the sake of discussion), importing the full dataset ("District Outpatient monthly

Re: [Dhis2-users] Mydatamart issue?

2012-05-07 Thread Ola Hodne Titlestad
Hi Andrew, Do you know how many rows that is returned from your pivot source query? The mydatmart tool is designed for what Jason describes in option 1) in his email. Also note that we require Excel 2007 or newer for pivot tables to work properly with mydatamart (on metatdata updates and re-link

Re: [Dhis2-users] DHIS version 2.8 released

2012-05-07 Thread Lars Helge Øverland
Hello, I did a test here with a CSV file containing 50 000 records and it took 16 minutes. I suspect this has to do with your Java configuration - have you e.g. set the environment variable JAVA_OPTS to allocate memory to Java ? A reasonable value would be -Xms500m -Xmx1000m -XX:PermSize=250m -

Re: [Dhis2-users] Mydatamart issue?

2012-05-07 Thread Jason Pickering
Hi Andrew, I do not know all of the circumstances of your data, but in general, Excel is limited by a certain number of rows and columns. I think Excel 2003 is limited to around 65,000 rows and Excel 2007 to over 1 million rows. PivotTables are limited by the amount of available memory on your mach

[Dhis2-users] Mydatamart issue?

2012-05-07 Thread Muhire Andrew
Hi to all, In Rwanda we have 4 months data in our DHIS2 database, And downloading data for the whole 4 months, Excel gives an overloading error or can even burst...Any advice on using mydatamart?   how are you running it over there? ..This issue happens when we download