Re: Best way/tool to debug memory leaks in HiveServer2

2018-03-14 Thread rajan sthapit
I tried getting the histo as you suggested jmap -histo:live ${PID} However, it only shows the histo of the live objects https://gist.github.com/anonymous/669eae8128c9a11f72fcdc1f82270460. Also the total memory from the histogram shows to be around 5GB. However, if I look at the total process m

Re: Best way/tool to debug memory leaks in HiveServer2

2018-03-13 Thread Gopal Vijayaraghavan
> It also shows that the process is consuming more than 30GB. However, it is > not clear what is causing the process to consume more than 30GB. The Xmx only applies to the heap size, there's another factor that is usually ignored which are the network buffers and compression buffers used by Java

Re: Best way/tool to debug memory leaks in HiveServer2

2018-03-13 Thread Prasanth Jayachandran
One option to check for is if THP is enabled on the node running HS2. If enable try disabling THP and see if it helps # Disable THP echo never | sudo tee -a /sys/kernel/mm/transparent_hugepage/enabled echo never | sudo tee -a /sys/kernel/mm/transparent_hugepage/defrag Thanks Prasanth On Mar 13