Re: Orphaned chunks

2015-04-17 Thread Flavio Pompermaier
I just set the -Xmx in the VM parameters to 10g and I have 8 virtual cores (4 physical). On Fri, Apr 17, 2015 at 2:48 PM, Robert Metzger wrote: > How much memory are you giving to each Flink TaskManager? > > On Fri, Apr 17, 2015 at 2:05 PM, Flavio Pompermaier > wrote: > >> Hi Robertm >> I forgo

Re: Orphaned chunks

2015-04-17 Thread Robert Metzger
How much memory are you giving to each Flink TaskManager? On Fri, Apr 17, 2015 at 2:05 PM, Flavio Pompermaier wrote: > Hi Robertm > I forgot to update about this error. > The root cause was an OOM cause by Jena RDF serialization that was causing > the failing of the entire job. > I also > https:

Re: Orphaned chunks

2015-04-17 Thread Flavio Pompermaier
Hi Robertm I forgot to update about this error. The root cause was an OOM cause by Jena RDF serialization that was causing the failing of the entire job. I also https://stackoverflow.com/questions/29660894/jena-thrift-serialization-oom-due-to-gc-overhead I created a thread on StackOverflow, let's s

Re: Orphaned chunks

2015-04-15 Thread Flavio Pompermaier
Yes Robert, Unfortunately I discovered that the error was caused by Phoenix just a little bit later the mail sending. The error is generated in the finalize() method of Pheonix MemoryManager so it seems somehow related to gc. I rerun the experiment logging to a file so I can investigate deeper the

Re: Orphaned chunks

2015-04-15 Thread Robert Metzger
Hey Flavio, I was not able to find the String "Orphaned chunk" in the Flink code base. However, I found it here: https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/memory/GlobalMemoryManager.java#L157 Maybe you've send the message to the wrong mailing list

Orphaned chunks

2015-04-15 Thread Flavio Pompermaier
Hi to all, another error today :( My job ended with a lot of "Orphaned chunk of bytes found during finalize". What could be the cause of this error? Best, Flavio