While batch write iceberg from spark why cannot keeping multiple files open.

2020-01-08 Thread xiaokun ding
Hi Ryan, aokolnychyi: I find in Writer.java#L506 , that should group data with the partition key before writing data into iceberg file. May I know your consideration for

Re: Apache Hive integration

2020-01-08 Thread Ryan Blue
Thanks for the interest in Hive integration! I haven't heard about progress here lately, so it's good that you bring it up. Hopefully the other people that are interested can jump in with their current status. I think you're right that the MR input and output formats are a good place to start, but

Apache Hive integration

2020-01-08 Thread Elliot West
Hello, We're considering working on an integration of Iceberg with Apache Hive, initially so that the latest snapshot of Iceberg tables can be queried via Hive, but later to allow the writing of data using the Iceberg table format. I wanted to first check for the existence and status of any simil