Re: Spec changes for deletion vectors

2024-10-14 Thread rdb...@gmail.com
> I think it might be worth mentioning the current proposal makes some, mostly minor, design choices to try to be compatible with Delta Lake deletion vectors. Yes it does, and thanks for pointing this out, Micah. I think it's important to consider whether compatibility is important to this communi

Re: [DISCUSS] [PyIceberg] Use of asserts to "programming the negative space"

2024-10-14 Thread André Luis Anastácio
Thank you Piotr Findeisen and Sung Yun, for your insights. I did a quick search and didn’t find anything more "pythonic." We could just use an if statement with raise, but I have some mixed feelings about that. Maybe we could create a precondition module with a function or a decorator. What do

Re: [VOTE] Table V3 Spec: Row Lineage

2024-10-14 Thread Russell Spitzer
Vote has completed and the Proposal is approved +1 Votes From Steven Wu Yufei Gu Amogh Jahagirdar Jack Ye Daniel Weeks Eduard Tudenhöfner Jean-Baptiste Onofré On Thu, Oct 10, 2024 at 8:11 PM Steven Wu wrote: > +1 > > On Thu, Oct 10, 2024 at 2:52 PM Yufei Gu wrote: > >> +1 >> Yufei >> >> >> O

Re: [EXTERNAL] Re: [DISCUSS] Column to Column filtering

2024-10-14 Thread Baldwin, Jennifer
Thanks everyone for your input, the use case in the document is something that I made up to describe how the filtering could apply. It is not a real world use case. I am being careful to not share our customer’s specific use case but it does use dates. 😊 I’ll see what I can come up with to pro

Re: [DISCUSS] REST: Refreshing vended credentials

2024-10-14 Thread Yufei Gu
Hi Eduard, Thanks for the proposal. I'm excited about the new spec. I have two questions: 1. This is probably a dumb question due to the lack of context, but I'm a bit confused about how clients should select a prefix to use. In scenarios where multiple prefixes exist, which one should the client

Re: [Discuss] LZ4 compression in Puffin spec

2024-10-14 Thread Piotr Findeisen
Micah, LZ4 framed format is actually not very complicated & well documented. If someone has cycles, https://github.com/airlift/aircompressor/pull/142 has most of the work done. It would be better to use an existing frame format over implementing yet another one. Best Piotr On Sun, 13 Oct 202