Re: Changelog scan for table with delete files

Thanks, Peter and Wing Yew Poon, for tackling these! I’ve been eager to review, but this week has been hectic. I plan to check out PR #10935 next week, though I’d be happy if someone beats me to it. Yufei On Mon, Sep 30, 2024 at 3:02 AM Péter Váry wrote: > Hi Team, > > The Changelog scan Java

Re: [DISCUSS] Iceberg Summit 2025 ?

+1 for a hybrid event John Zhuge On Mon, Sep 30, 2024 at 10:12 AM Sung Yun wrote: > Hi JB, thank you for starting this thread! > > I’m already very excited at the prospect of connecting with other members > of the community. I think it would be nice to organize one next year given > the succe

Re: [VOTE] Table v3 spec: Add unknown and new type promotion

I'm -0.0 as worded currently. I think there are some more aspects that should be defined for date->timestamp/timestamp_ns promotion (left comments on the PR). The addition of an Unknown type seems like a good addition. Thanks, Micah On Mon, Sep 30, 2024 at 2:32 PM Yufei Gu wrote: > +1(binding

Re: [DISCUSS] Define calendar used in specification?

I just wanted to follow up on this. A compromise on language here could be that Iceberg uses ISO8601 calendar. For dates prior to the Julien/Gregorian calendar, implementations are encouraged to use proleptic-gregorian but this is left unspecified by the specification. Thoughts? Micah On T

Re: [VOTE] Table v3 spec: Add unknown and new type promotion

+1(binding) Yufei On Mon, Sep 30, 2024 at 12:42 PM Amogh Jahagirdar <2am...@gmail.com> wrote: > +1 (binding) > > Thanks, > Amogh Jahagirdar > > On Mon, Sep 30, 2024 at 1:39 PM rdb...@gmail.com wrote: > >> +1 (binding) >> >> On Mon, Sep 30, 2024 at 12:32 PM Daniel Weeks wrote: >> >>> +1 (bindi

Re: [Discuss] Geospatial Support

I have a couple of comments that I'd like to see addressed. First, I think that the definition of the bounding box needs to be more clear: the bounding box must include all points that lie on an object's edges or within an object. If that isn't required then we can't use the bounding box for filte

Re: [Discuss] Geospatial Support

Thanks Szehon! My comments were addressed. I'm ready to vote. Yufei On Mon, Sep 30, 2024 at 11:47 AM Russell Spitzer wrote: > All my concerns are addressed, I'm ready to vote. > > On Mon, Sep 30, 2024 at 1:21 PM Szehon Ho wrote: > >> Hi all, >> >> There have been several rounds of discussion

Re: [EXTERNAL] Re: [DISCUSS] Column to Column filtering

It has come to my attention that there was no attachment. I have created google doc instead. Thanks. https://docs.google.com/document/d/1HZa3AyPPfgz9VOVA9rPhJJ8f3F-3tEel_53nIlvYlo0/edit?usp=sharing From: Baldwin, Jennifer Date: Friday, September 27, 2024 at 12:54 PM To: dev@iceberg.apache.org

Re: [VOTE] Table v3 spec: Add unknown and new type promotion

+1 (binding) Thanks, Amogh Jahagirdar On Mon, Sep 30, 2024 at 1:39 PM rdb...@gmail.com wrote: > +1 (binding) > > On Mon, Sep 30, 2024 at 12:32 PM Daniel Weeks wrote: > >> +1 (binding) >> >> On Fri, Sep 27, 2024 at 2:41 PM Russell Spitzer < >> russell.spit...@gmail.com> wrote: >> >>> +1 (bindin

Re: [VOTE] Table v3 spec: Add unknown and new type promotion

+1 (binding) On Mon, Sep 30, 2024 at 12:32 PM Daniel Weeks wrote: > +1 (binding) > > On Fri, Sep 27, 2024 at 2:41 PM Russell Spitzer > wrote: > >> +1 (binding) >> >> On Fri, Sep 27, 2024 at 4:37 PM rdb...@gmail.com >> wrote: >> >>> Hi everyone, >>> >>> I'd like to vote on PR #10955 >>>

Re: [VOTE] Table v3 spec: Add unknown and new type promotion

+1 (binding) On Fri, Sep 27, 2024 at 2:41 PM Russell Spitzer wrote: > +1 (binding) > > On Fri, Sep 27, 2024 at 4:37 PM rdb...@gmail.com wrote: > >> Hi everyone, >> >> I'd like to vote on PR #10955 >> that has been open for a >> while with the chang

Re: [Discuss] Geospatial Support

All my concerns are addressed, I'm ready to vote. On Mon, Sep 30, 2024 at 1:21 PM Szehon Ho wrote: > Hi all, > > There have been several rounds of discussion on the PR: > https://github.com/apache/iceberg/pull/10981 and I think most of the main > points have been addressed. > > If anyone is inte

Re: [Discuss] Geospatial Support

Hi all, There have been several rounds of discussion on the PR: https://github.com/apache/iceberg/pull/10981 and I think most of the main points have been addressed. If anyone is interested, please take a look. If there are no other major points, we plan to start a VOTE thread soon. I know Jia

Re: [DISCUSS] Modify ThreadPools.newWorkerPool to avoid unnecessary Shutdown Hook registration

+1 for `newExitingWorkerPool`. On Fri, Sep 27, 2024 at 4:23 PM Steven Wu wrote: > > I don't think that solves the problem that these are used more widely > than intended and without people knowing the behavior. > > Ryan, to solve this problem, I suggest we deprecate the current > `newWorkerPool

Re: [DISCUSS] Iceberg Summit 2025 ?

Hi JB, thank you for starting this thread! I’m already very excited at the prospect of connecting with other members of the community. I think it would be nice to organize one next year given the success of the 2024 Summit. > should we have another event ? Yes! Definitely > would you like there

Re: [DISCUSS] Iceberg Summit 2025 ?

+1 to hybrid event with an in-person element. Things I like to see: * Real-world experience from companies running Iceberg at scale * Iceberg catalogs and how it's used * Integrations with open-source projects in the broader ecosystem * Forward-looking statements for the direction of the Iceberg e

Re: Clarification on DayTransform Result Type

Thank you both for the insights and context. As Russell pointed out, the "day partition transform" result is true of int type. The Types.DateType correspo

Re: [DISCUSS] Iceberg Summit 2025 ?

Thank you, JB, for taking the initiative to get the conversation started for the next Iceberg Summit! I’m really excited to see the community considering a hybrid event for 2025. Having the option for in-person interaction would definitely enhance the sense of connection among contributors and us

Changelog scan for table with delete files

Hi Team, The Changelog scan Java API interfaces were created a long time ago by Anton, but it has not been implemented until yet. There is a Spark specific SQL implementation for the feature, but the feature is not available on the Java API. The Flink CDC streaming read is one of the often requir