Iceberg at Subsurface Conference

2020-07-08 Thread Mass Dosage
Hello all, You might be interested to know that myself and Christine Mathiesen will be presenting our work on adding Hive read support to Iceberg at the upcoming Subsurface Cloud Data Lake conference. The talk is entitled "Hiveberg: Integrating Apache Iceberg with the Hive Metastore". You can regi

Re: [DISCUSS] July board report

2020-07-08 Thread Mass Dosage
LGTM! On Tue, 7 Jul 2020 at 21:27, Ryan Blue wrote: > Hi everyone, > > Here's my draft report for July. Feel free to comment and suggest updates > that I've missed. Thanks! > > rb > > ## Description: > Apache Iceberg is a table format for huge analytic datasets that is > designed > for high perf

Re: Arrow Support in Parquet Writers

2020-07-08 Thread Ryan Blue
Yes, I think there is a reasonable path to an implementation that doesn't require the Iceberg API. While the first step is just getting it working, I think we could refactor and remove the dependency on iceberg-api and iceberg-parquet. Then we would have a module that can be used independently. Th

Re: Custom Metastore in Presto Connector

2020-07-08 Thread Ryan Blue
I don't think that the Presto connector currently supports custom catalog implementations without building a connector based on Iceberg. You might check with the Presto community on their mailing list or Slack to find out what they recommend here. On Mon, Jul 6, 2020 at 7:27 PM eskrm wrote: > Hi

Re: [DISCUSS] July board report

2020-07-08 Thread Julien Le Dem
+1 On Wed, Jul 8, 2020 at 10:25 AM Mass Dosage wrote: > LGTM! > > On Tue, 7 Jul 2020 at 21:27, Ryan Blue wrote: > >> Hi everyone, >> >> Here's my draft report for July. Feel free to comment and suggest updates >> that I've missed. Thanks! >> >> rb >> >> ## Description: >> Apache Iceberg is a ta

Update on Hive InputFormat

2020-07-08 Thread Adrien Guillo
Hey Iceberg devs, Sorry I wasn't able to make today's sync and provide you with an update. I've made good progress on implementing a Hive InputFormat that wraps the MR v2 InputFormat. The branch I'm working on passes the test suite that Adrian shared with me last week. I'll push a PR shortly. My u