Re: offheap memory allocation and memory leak bug

2016-06-20 Thread Till Rohrmann
That would be great. Best you directly post the link to the JIRA issue. Cheers, Till On Mon, Jun 20, 2016 at 12:55 PM, CPC wrote: > Hi Till, > > I saw jira issue. Do you want me to upload input dataset as well? If you > want i can prepare a github repo if it would be more easier. > On Jun 20, 2

Re: offheap memory allocation and memory leak bug

2016-06-20 Thread CPC
Hi Till, I saw jira issue. Do you want me to upload input dataset as well? If you want i can prepare a github repo if it would be more easier. On Jun 20, 2016 1:10 PM, "Till Rohrmann" wrote: > Hi, > > your observation sounds like a bug to me and we have to further investigate > it. I assume that

Re: offheap memory allocation and memory leak bug

2016-06-20 Thread Till Rohrmann
Hi, your observation sounds like a bug to me and we have to further investigate it. I assume that you’re running a batch job, right? Could you maybe share your complete configuration and the job to reproduce the problem with us? I think that your investigation that direct buffers are not properly

Re: offheap memory allocation and memory leak bug

2016-06-19 Thread CPC
Hi, I think i found some information regarding this behavior. In jvm it is almost imposible to free allocated memory via ByteBuffer.allocateDirect. There is no explicit way to say jvm "free this direct bytebuffer". In some forums they said you can free memory with below method: > def releaseBuff

Re: offheap memory allocation and memory leak bug

2016-06-18 Thread CPC
Hello, I repeated the same test with conf values. > taskmanager.heap.mb: 6500 > > taskmanager.memory.off-heap: true > > taskmanager.memory.fraction: 0.9 > > i set TM_MAX_OFFHEAP_SIZE="6G" in taskmanager sh. Taskmanager started with > capacman 14543 323 56.0 17014744 13731328 pts/1 Sl 16:23 35:

offheap memory allocation and memory leak bug

2016-06-17 Thread CPC
Hi, I am making some test about offheap memory usage and encounter an odd behavior. My taskmanager heap limit is 12288 Mb and when i set "taskmanager.memory.off-hep:true" for every job it allocates 11673 Mb off heap area at most which is heapsize*0.95(value of taskmanager.memory.fraction). But whe