I think Lars is probably the definitive source here, but let me offer my views.
I think this has nothing to do with the database (MySQL or Postgresql) but rather how DHIS2 does the aggregation. A big chunk of data will get pulled into memory. If the data is too big and the memory is not enough, this error will result. I have seen it many times. We have had similar issues with our database, when aggregating historical data in one chunk. So, instead, I simply do it year by year, piece by piece. This is fairly easy to automate with a few CURL calls. We are using Postgres, and have had similar issues, which have been solved by 1) Increasing the PermSpace parameter 2) Aggregate less data at a time. Switching to Postgres may have some other advantage, but I do not think it will solve this issue. Properly tuned, it should work fine. Regards, Jason On Tue, Sep 11, 2012 at 4:06 PM, Hannan Khan <hann...@gmail.com> wrote: > Dear Knut > > Actually I never use PostgreSQL. So I asked few of friends and they > suggests me that if the application are tuned and used in MySQL then stay > with that. > > As you are suggesting to use PostgreSQL can you suggest how I could I > shift database from MySQL to PostgreSQL? > > Regards > > Hannan > > > On Tue, Sep 11, 2012 at 4:29 PM, Knut Staring <knu...@gmail.com> wrote: > >> Perhaps this is a case where it would make sense to switch to >> PostgreSQL and possibly also make use of more of the internal >> crunching capabilities of the database? >> >> Knut >> >> On Tue, Sep 11, 2012 at 10:09 AM, Hannan Khan <hann...@gmail.com> wrote: >> > I might found an alternative solution. Yesterday we run data mart for 9 >> > month and it was completed successfully but when we run for 5 years, the >> > errors arrived. >> > >> > Today and trying with again and get back to you all with the result. But >> > when we run datamart for 9 moth it takes 34 minutes, which is long is >> not >> > it? >> > >> > Regards >> > >> > Hannan >> > >> > >> > On Tue, Sep 11, 2012 at 2:01 PM, Morten Olav Hansen <morte...@gmail.com >> > >> > wrote: >> >> >> >> That's a really big database... I think maybe Lars should answer if the >> >> memory of the server is large enough (I'm sure he can compare it with >> what >> >> he has been doing to tune the Kenya server). >> >> >> >> -- >> >> Morten >> >> >> >> >> >> >> >> On Tue, Sep 11, 2012 at 2:59 PM, Hannan Khan <hann...@gmail.com> >> wrote: >> >>> >> >>> Hi Morten >> >>> >> >>> Server have 8GB memory and database is 3159MB. We are using MySQL. >> >>> >> >>> Regards >> >>> >> >>> Hannan >> >>> >> >>> >> >>> On Tue, Sep 11, 2012 at 1:53 PM, Morten Olav Hansen < >> morte...@gmail.com> >> >>> wrote: >> >>>> >> >>>> Hi >> >>>> >> >>>> How much memory does your server have? and what is the size of your >> >>>> database? >> >>>> >> >>>> -- >> >>>> Morten >> >>>> >> >>>> >> >>>> >> >>>> On Tue, Sep 11, 2012 at 2:51 PM, Hannan Khan <hann...@gmail.com> >> wrote: >> >>>>> >> >>>>> -XX:MaxPermSize=2048m and -XX:PermSize=512m >> >>>>> >> >>>>> >> >>>>> On Tue, Sep 11, 2012 at 11:27 AM, Jason Pickering >> >>>>> <jason.p.picker...@gmail.com> wrote: >> >>>>>> >> >>>>>> The PergGen space is the important option. What are you >> JAVA_OPTS.? >> >>>>>> >> >>>>>> -- >> >>>>>> Sent from my mobile >> >>>>>> >> >>>>>> On Sep 11, 2012 7:25 AM, "Hannan Khan" <hann...@gmail.com> wrote: >> >>>>>>> >> >>>>>>> Thanks Jason. >> >>>>>>> >> >>>>>>> I put the Java heap space 1GB and Maximum 6GB (Xmx6144m and >> >>>>>>> Xms1024m). But still getting 'java.lang.OutOfMemoryError: Java >> heap space' >> >>>>>>> error. That why I asked that whether that is a bug or not. >> >>>>>>> >> >>>>>>> I am trying to other options and contact you soon. >> >>>>>>> >> >>>>>>> Regards >> >>>>>>> >> >>>>>>> Hannan >> >>>>>>> >> >>>>>>> On Mon, Sep 10, 2012 at 11:40 AM, Jason Pickering >> >>>>>>> <jason.p.picker...@gmail.com> wrote: >> >>>>>>>> >> >>>>>>>> Hi Hannan, >> >>>>>>>> >> >>>>>>>> You configuration may have worked previously, but some things >> were >> >>>>>>>> changed in the datamart and you will need to allocate more >> memory to >> >>>>>>>> it after 2.7. >> >>>>>>>> >> >>>>>>>> The details are here of the server setup are here >> >>>>>>>> >> >>>>>>>> >> http://dhis2.org/doc/snapshot/en/implementer/html/ch08.html#d5e413 >> >>>>>>>> >> >>>>>>>> and the important part is the JAVA_OPTS environment variable, for >> >>>>>>>> instance... >> >>>>>>>> >> >>>>>>>> export JAVA_OPTS='-Xmx6000m -Xms3000m -XX:MaxPermSize=800m >> >>>>>>>> -XX:PermSize=400m' >> >>>>>>>> >> >>>>>>>> I am not really sure what your settings are, but you should at >> least >> >>>>>>>> specify these options, of course adjusting them for the amount of >> >>>>>>>> memory which can be allocated to DHIS2 . >> >>>>>>>> >> >>>>>>>> Best regards, >> >>>>>>>> Jason >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On Mon, Sep 10, 2012 at 10:49 AM, Hannan Khan <hann...@gmail.com >> > >> >>>>>>>> wrote: >> >>>>>>>> > Dear Jason >> >>>>>>>> > >> >>>>>>>> > Thanks for your advice. We are running dhis2 from 2010. In >> earlier >> >>>>>>>> > version >> >>>>>>>> > we dont have any problem creating datamart. After updating to >> >>>>>>>> > version 2.7 we >> >>>>>>>> > imported a big chuck of data from two data souce and after >> that we >> >>>>>>>> > have >> >>>>>>>> > problem creating datamart. >> >>>>>>>> > >> >>>>>>>> > So far I know the configuration is ok, however I will check >> again >> >>>>>>>> > and get >> >>>>>>>> > back to you. >> >>>>>>>> > >> >>>>>>>> > Regards >> >>>>>>>> > >> >>>>>>>> > Hannan >> >>>>>>>> > from Samsung Galaxy >> >>>>>>>> > >> >>>>>>>> > On Sep 9, 2012 5:39 PM, "Jason Pickering" >> >>>>>>>> > <jason.p.picker...@gmail.com> >> >>>>>>>> > wrote: >> >>>>>>>> >> >> >>>>>>>> >> This is not a bug. You must increase the heap size. Please >> >>>>>>>> >> consult the >> >>>>>>>> >> implementation manual for details. >> >>>>>>>> >> >> >>>>>>>> >> Regards, >> >>>>>>>> >> Jason >> >>>>>>>> >> >> >>>>>>>> >> -- >> >>>>>>>> >> Sent from my mobile >> >>>>>>>> >> >> >>>>>>>> >> On Sep 9, 2012 1:12 PM, "Hannan Khan" <hann...@gmail.com> >> wrote: >> >>>>>>>> >>> >> >>>>>>>> >>> We are using dhis2 version 2.7. Whenever we generating >> datamart >> >>>>>>>> >>> are it >> >>>>>>>> >>> taking long time and it never ends.The error message is (in >> the >> >>>>>>>> >>> tomcat log >> >>>>>>>> >>> file) >> >>>>>>>> >>> >> >>>>>>>> >>> >> 'dhis2-org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor >> >>>>>>>> >>> processChildren SEVERE: Exception invoking periodic >> operation: >> >>>>>>>> >>> java.lang.OutOfMemoryError: Java heap space'. >> >>>>>>>> >>> >> >>>>>>>> >>> I attached the log file. >> >>>>>>>> >>> >> >>>>>>>> >>> Regards >> >>>>>>>> >>> >> >>>>>>>> >>> Hannan Khan >> >>>>>>>> >>> >> >>>>>>>> >>> National Data Warehouse >> >>>>>>>> >>> >> >>>>>>>> >>> >> >>>>>>>> >>> >> >>>>>>>> >>> >> >>>>>>>> >>> _______________________________________________ >> >>>>>>>> >>> Mailing list: https://launchpad.net/~dhis2-devs >> >>>>>>>> >>> Post to : dhis2-devs@lists.launchpad.net >> >>>>>>>> >>> Unsubscribe : https://launchpad.net/~dhis2-devs >> >>>>>>>> >>> More help : https://help.launchpad.net/ListHelp >> >>>>>>>> >>> >> >>>>>>>> > >> >>>>>>> >> >>>>>>> >> >>>>> >> >>>>> >> >>>>> _______________________________________________ >> >>>>> Mailing list: https://launchpad.net/~dhis2-devs >> >>>>> Post to : dhis2-devs@lists.launchpad.net >> >>>>> Unsubscribe : https://launchpad.net/~dhis2-devs >> >>>>> More help : https://help.launchpad.net/ListHelp >> >>>>> >> >>>> >> >>> >> >> >> > >> > >> > _______________________________________________ >> > Mailing list: https://launchpad.net/~dhis2-devs >> > Post to : dhis2-devs@lists.launchpad.net >> > Unsubscribe : https://launchpad.net/~dhis2-devs >> > More help : https://help.launchpad.net/ListHelp >> > >> >> >> >> -- >> Knut Staring >> Dept. of Informatics, University of Oslo >> +4791880522 >> http://dhis2.org >> > > > _______________________________________________ > Mailing list: https://launchpad.net/~dhis2-devs > Post to : dhis2-devs@lists.launchpad.net > Unsubscribe : https://launchpad.net/~dhis2-devs > More help : https://help.launchpad.net/ListHelp > >
_______________________________________________ Mailing list: https://launchpad.net/~dhis2-devs Post to : dhis2-devs@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
