Hi, I need some inputs to execute hive queries in parallel. I tried doing this using CLI (by opening multiple ssh connection) and executed 4 HQL's; it was observed that the queries are getting executed sequentially. All the FOUR queries got submitted however while the first one was in execution mode the other were in pending state. I was performing this activity on the EMR running on Batch mode hence didn't able to dig into the logs.
The hive CLI uses native hive connection which by default uses the FIFO scheduler. This might be one of the reason for the queries getting executed in sequence. I also observed that when multiple queries are executed using multiple HUE sessions, it provides the parallel execution functionality. Can you please suggest how the functionality of HUE can be replicated using CLI? I am aware of beeswax client however i am not sure how this can be used during EMR- batch mode processing. Thanks in advance for going through this. Kindly let me know your thoughts on the same.
