Hi everyone, I have a cluster where each container is configured at 4GB and some of my queries are getting over in 30 to 40 seconds. This leads me to believe that I have too much memory for my containers and I am thinking of reducing the container size to 1.5GB(hive.tez.container.size) but I am looking for a few more concrete data points to find out if really I have oversized containers? I looked into the tez view of my DAG and the counters give me: PHYSICAL_MEMORY_BYTES 907965628416 VIRTUAL_MEMORY_BYTES 1560263561216 I am guessing this is wrong as there is no way the query could finish in 20 seconds on a 98GB cluster if the actual memory required by the query is 907GB. Any help to find some data points regarding determination of oversized containers is very much appreciated!
Thanks Ranjan
