Re: SparkR integration with Hive 3 spark-r

2019-11-24 Thread Felix Cheung
. From: Alfredo Marquez Sent: Friday, November 22, 2019 4:26:49 PM To: user@spark.apache.org Subject: Re: SparkR integration with Hive 3 spark-r Does anyone else have some insight to this question? Thanks, Alfredo On Mon, Nov 18, 2019, 3:00 PM Alfredo Marquez

Re: SparkR integration with Hive 3 spark-r

2019-11-22 Thread Alfredo Marquez
Does anyone else have some insight to this question? Thanks, Alfredo On Mon, Nov 18, 2019, 3:00 PM Alfredo Marquez wrote: > Hello Nicolas, > > Well the issue is that with Hive 3, Spark gets it's own metastore, > separate from the Hive 3 metastore. So how do you reconcile this > separation of

Re: SparkR integration with Hive 3 spark-r

2019-11-18 Thread Alfredo Marquez
Hello Nicolas, Well the issue is that with Hive 3, Spark gets it's own metastore, separate from the Hive 3 metastore. So how do you reconcile this separation of metastores? Can you continue to "enableHivemetastore" and be able to connect to Hive 3? Does this connection take advantage of Hive's L

Re: SparkR integration with Hive 3 spark-r

2019-11-18 Thread Nicolas Paris
Hi Alfredo my 2 cents: To my knowlegde and reading the spark3 pre-release note, it will handle hive metastore 2.3.5 - no mention of hive 3 metastore. I made several tests on this in the past[1] and it seems to handle any hive metastore version. However spark cannot read hive managed table AKA tra