Thanks, Nathan. How much is should be in general? On Thu, Mar 5, 2015 at 10:15 AM, Nathan Leung <[email protected]> wrote:
> Your worker is allocated a maximum of 768mb of heap. It's quite possible > that this is not enough. Try increasing Xmx i worker.childopts. > On Mar 5, 2015 1:10 PM, "Sa Li" <[email protected]> wrote: > >> Hi, All >> >> I have been running a trident topology on production server, code is like >> this: >> >> topology.newStream("spoutInit", kafkaSpout) >> .each(new Fields("str"), >> new JsonObjectParse(), >> new Fields("eventType", "event")) >> .parallelismHint(pHint) >> .groupBy(new Fields("event")) >> .persistentAggregate(PostgresqlState.newFactory(config), new >> Fields("eventType"), new EventUpdater(), new Fields("eventWord")) >> ; >> >> Config conf = new Config(); >> conf.registerMetricsConsumer(LoggingMetricsConsumer.class, 1); >> >> Basically, it does simple things to get data from kafka, parse to different >> field and write into postgresDB. But in storm UI, I did see such error, >> "java.lang.OutOfMemoryError: GC overhead limit exceeded". It all happens in >> same worker of each node - 6703. I understand this is because by default the >> JVM is configured to throw this error if you are spending more than *98% of >> the total time in GC and after the GC less than 2% of the heap is recovered*. >> >> I am not sure what is exact cause for memory leak, is it OK by simply >> increase the heap? Here is my storm.yaml: >> >> supervisor.slots.ports: >> >> - 6700 >> >> - 6701 >> >> - 6702 >> >> - 6703 >> >> nimbus.childopts: "-Xmx1024m -Djava.net.preferIPv4Stack=true" >> >> ui.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true" >> >> supervisor.childopts: "-Djava.net.preferIPv4Stack=true" >> >> worker.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true" >> >> >> Anyone has similar issues, and what will be the best way to overcome? >> >> >> thanks in advance >> >> AL >> >> >> >>
