Vladi,
We have similarities to what you describe.
While I agree that what Owen mentioned about an implementation of Hive's
Rawstore API will be really useful, I don't believe it fully answers Vladi's
question. I think the main concern here is smooth migration of existing clients
to iceberg tables and what happens in a hybrid world when all the tables are
not fully switched to iceberg? @Owen O'Malley, @rdb...@netflix.com do you have
any thoughts on this?
I have an additional concern: Operations like: "listPartitions" that are a
regular feature in Hive Metastore Client API don't appear to be very
straightforward in Iceberg. Doesn't Presto or other clients used to getting a
list of partitions? Any thoughts on these would be very helpful.
Thanks,
Arvind
On 1/7/19, 1:19 PM, "Owen O'Malley" <owen.omal...@gmail.com> wrote:
The group has moved to the Apache infrastructure, so we should use
dev@iceberg.apache.org .
What is required, but not started, is for someone to implement Hive's
RawStore API with an Iceberg backend. That would let you use Hive SQL
commands to manipulate the Iceberg tables.
.. Owen
On Mon, Jan 7, 2019 at 1:01 PM 'Vladi Feigin' via Iceberg Developers <
iceberg-de...@googlegroups.com> wrote:
> Hello ,
>
> I still confused a bit how Iceberg interacts with Hive (metastore).
> In our case we have many Hive tables and a lot Spark and Presto jobs
> reading, creating, writing to Hive
> Moving to Iceberg, even gradually raising a few questions :
> 1. Are new tables created via Iceberg visible (by sparlk/presto) in Hive
> metastore as well?
> 2. Should we migrate somehow existing Hive tables to be supported by
> Iceberg?
> 3. Is there any impact on the existing (spark,presto) jobs when moving to
> Iceberg?
>
> I understand that creating a new system from scratch with Iceberg is
> probably easier comparing to the projects heavily using Hive metastore but
> this is the use case in a lot of projects nowdays
> Thank you
> Vladi Feigin
>
> --
> You received this message because you are subscribed to the Google Groups
> "Iceberg Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to iceberg-devel+unsubscr...@googlegroups.com.
> To post to this group, send email to iceberg-de...@googlegroups.com.
> To view this discussion on the web visit
>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Ficeberg-devel%2F5d38541c-f73f-471f-b8db-5430238c4376%2540googlegroups.com&data=02%7C01%7Capruthi%40linkedin.com%7C0ff7ac0a61c94a5572ed08d674e5c501%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636824927540936290&sdata=%2BDoc4lEFexkeCipgBmoht8ZfEhfi3beQmXmNguxALYU%3D&reserved=0
>
<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Ficeberg-devel%2F5d38541c-f73f-471f-b8db-5430238c4376%2540googlegroups.com%3Futm_medium%3Demail%26utm_source%3Dfooter&data=02%7C01%7Capruthi%40linkedin.com%7C0ff7ac0a61c94a5572ed08d674e5c501%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636824927540936290&sdata=jCCxuMCjvgswEJQv0twAAd1IrxbLIjhfpKy8cgoiypg%3D&reserved=0>
> .
> For more options, visit
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Foptout&data=02%7C01%7Capruthi%40linkedin.com%7C0ff7ac0a61c94a5572ed08d674e5c501%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636824927540936290&sdata=i0%2BMLZRVy50HywHfxIA%2FsWxlTXn%2B74UXBPTHlZJ7RRA%3D&reserved=0.
>
