Re: Best way to Hive to Spark migration

Ok this is not much detail, but you are probably best off if you migrate them to SparkSQL. Depends also on the Hive version and Spark version. If you have a recent one with TEZ+llap I would not expect so much difference. It can be also less performant -Spark SQL got only recently some features

Re: Best way to Hive to Spark migration

Hi I have lot of ETL jobs (complex ones) , since they are SLA critical , I am planning them to migrate to spark. On Thu, Apr 5, 2018 at 10:46 AM, Jörn Franke wrote: > You need to provide more context on what you do currently in Hive and what > do you expect from the migration. > > On 5. Apr 201

Re: Best way to Hive to Spark migration

You need to provide more context on what you do currently in Hive and what do you expect from the migration. > On 5. Apr 2018, at 05:43, Pralabh Kumar wrote: > > Hi Spark group > > What's the best way to Migrate Hive to Spark > > 1) Use HiveContext of Spark > 2) Use Hive on Spark > (https://

Best way to Hive to Spark migration

Hi Spark group What's the best way to Migrate Hive to Spark 1) Use HiveContext of Spark 2) Use Hive on Spark ( https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started ) 3) Migrate Hive to Calcite to Spark SQL Regards

time for Apache Spark 3.0?

There was a discussion thread on scala-contributors about Apache Spark not yet supporting Scala 2.12, and that got me to think perhaps it is about time for Spark to work towards the 3.0 release. By the t

Re: Hadoop 3 support

What would be the strategy with hive? Cherry pick patches? Update to more “modern” versions (like 2.3?) I know of a few critical schema evolution fixes that we could port to hive 1.2.1-spark _ From: Steve Loughran Sent: Tuesday, April 3, 2018 1:33 PM Subject: Re: H