RE: Oversized container estimation

2016-11-25 Thread Ranjan Banerjee
Hi Rajesh, Thanks a lot for the insight. When you mean CPU are you referring to the vcore of yarn? The yarn min container size(yarn-scheduler.minimum.allocation.mb) is set to 1.5GB and the minimum cores per container(yarn-scheduler-minimum.allocation.vcores) is set to 1. Are u saying that if

Re: Oversized container estimation

2016-11-25 Thread Rajesh Balamohan
Those are cumulative figures in the DAG level. You may want to check the gc logs emitted at task level to check the details on whether complete memory is used or not. Not sure what is the yarn-min container size specified in your cluster. But based on that, you may run into the risk of running too

Oversized container estimation

2016-11-25 Thread Ranjan Banerjee
Hi everyone, I have a cluster where each container is configured at 4GB and some of my queries are getting over in 30 to 40 seconds. This leads me to believe that I have too much memory for my containers and I am thinking of reducing the container size to 1.5GB(hive.tez.container.size) but I am

[GitHub] hive pull request #118: fix stripe size smaller than expected

2016-11-25 Thread melode11
GitHub user melode11 opened a pull request: https://github.com/apache/hive/pull/118 fix stripe size smaller than expected I saw there's a Jira item: https://issues.apache.org/jira/browse/HIVE-13232 moved the compressed = null out of if block. But that is not seem to be a complete f

[jira] [Created] (HIVE-15285) err info for itests mvn building is not incorrect

2016-11-25 Thread Fei Hui (JIRA)
Fei Hui created HIVE-15285: -- Summary: err info for itests mvn building is not incorrect Key: HIVE-15285 URL: https://issues.apache.org/jira/browse/HIVE-15285 Project: Hive Issue Type: Bug