Thanks Elliot ! Let me check it out !
On Mon, 17 Apr, 2023, 10:08 pm Elliot West, wrote:
> Hi Ankit,
>
> While not a part of Spark, there is a project called 'WaggleDance' that
> can federate multiple Hive metastores so that they are accessible via a
> single URI: https://github.com/ExpediaGroup
There is a DSv2-based Hive connector in Apache Kyuubi[1] that supports
connecting multiple HMS in a single Spark application.
Some limitations
- currently only supports Spark 3.3
- has a known issue when using w/ `spark-sql`, but OK w/ spark-shell and
normal jar-based Spark application.
[1]
http
Hi Ankit,
While not a part of Spark, there is a project called 'WaggleDance' that can
federate multiple Hive metastores so that they are accessible via a single
URI: https://github.com/ExpediaGroup/waggle-dance
This may be useful or perhaps serve as inspiration.
Thanks,
Elliot.
On Mon, 17 Apr
++
User Mailing List
Just a reminder, anyone who can help on this.
Thanks a lot !
Ankit Prakash Gupta
On Wed, Apr 12, 2023 at 8:22 AM Ankit Gupta wrote:
> Hi All
>
> The question is regarding the support of multiple Remote Hive Metastore
> catalogs with Spark. Starting Spark 3, multiple catal
Hi All
The question is regarding the support of multiple Remote Hive Metastore
catalogs with Spark. Starting Spark 3, multiple catalog support is added in
spark, but have we implemented any CatalogPlugin that can help us configure
multiple Remote Hive Metastore Catalogs ? If yes, can anyone help m
