Tuning webhcat?

2017-10-20 Thread Ying Chen
I am using webHCat to submit very simple queries via the /hive endpoints. Then I used the /jobs endpoint to query if the job has been completed or not (not via a callback from the hive endpoint). I am noticing that this loop takes a little over 1 minutes, Is there someway to tune this so that the

Re: In reduce task,i have a join operation ,and i found "org.apache.hadoop.mapred.FileInputFormat: Total input paths to process : 1" cast much long

2017-10-20 Thread Daniel Bruce
OK, more updates. Today I was running the query with Yarn and also turned on DEBUG logging. Here's what I found from the task log for the dangling task. I noticed that after the RowContainer has been created by Hive, there are a lot of IPC/RPC related logs (still printing), every 3 second apart. M

Re: In reduce task,i have a join operation ,and i found "org.apache.hadoop.mapred.FileInputFormat: Total input paths to process : 1" cast much long

2017-10-20 Thread Daniel Bruce
Moreover, the IP 10.224.174.71 is a different node, not the one executing the reduce task. Why did that happen? On Fri, Oct 20, 2017 at 3:37 PM, Daniel Bruce wrote: > OK, more updates. Today I was running the query with Yarn and also turned > on DEBUG logging. Here's what I found from the task l