Re: Running Hive on Spark

2019-03-13 Thread Rajesh Balamohan
"Hive on Spark" uses Spark purely as execution engine. It would not get the benefits of codegen and other optimizations of Spark. If it is mainly for testing, OOTB parameters should work without issues. However, Tez has lot better edge than Hive on Spark. Some of the areas where Hive on Spark ne

Re: Running Hive on Spark

2019-03-12 Thread Daniel Mateus Pires
Hi Rajesh, I'm trying to further my understanding of the various interactions and set-ups for Hive + Spark My understanding so far is that running queries against the SparkThriftServer uses the SparkSQL engine whereas the HiveServer2 + Hive + Spark execution engine uses Hive primitives and only u

Re: Running Hive on Spark

2019-03-11 Thread Rajesh Balamohan
Not sure why you are using SparkThriftServer. OOTB HiveServer2 would be good enough for this. Is there any specific reason for moving from tez to spark as execution engine? ~Rajesh.B On Mon, Mar 11, 2019 at 9:45 PM Daniel Mateus Pires wrote: > Hi there, > > I would like to run Hive using Spark