Re: Updates/Deletes/Upserts in Iceberg

2019-05-29 Thread Jacques Nadeau
Yeah, I totally forgot to record our discussion. Will do so next time, sorry. -- Jacques Nadeau CTO and Co-Founder, Dremio On Wed, May 29, 2019 at 4:24 PM Ryan Blue wrote: > It wasn't recorded, but I can summarize what we talked about. Sorry I > haven't sent this out earlier. > > We talked abou

Re: Updates/Deletes/Upserts in Iceberg

2019-05-29 Thread Ryan Blue
It wasn't recorded, but I can summarize what we talked about. Sorry I haven't sent this out earlier. We talked about the options and some of the background in Iceberg -- basically that it isn't possible to determine the order of commits before you commit so you can't rely on some monotonically inc

Re: Updates/Deletes/Upserts in Iceberg

2019-05-29 Thread Venkatakrishnan Sowrirajan
Hi Ryan, I couldn't attend the meeting. Just curious, if this is recorded by any chance. Regards Venkata krishnan On Fri, May 24, 2019 at 8:49 AM Ryan Blue wrote: > Yes, I agree. I'll talk a little about a couple of the constraints of this > as well. > > On Fri, May 24, 2019 at 5:52 AM Anton

Re: Future of Iceberg Parquet Reader

2019-05-29 Thread Ryan Blue
Am I correct that performance is the main reason to have a custom reader in Iceberg? Are there any other purposes? A common question I get is why not improve parquet-mr instead of writing a new reader? I know that almost every system that cares about performance has its own reader, but why so? Per