[DISCUSS] Optimize for CBO

2024-09-23 Thread Xingyuan Lin
Hi Iceberg dev team, Cost-based optimizers need these stats: - record count - null count - number of distinct values (NDVs) - min/max values - column sizes Today, to get these stats, an engine must process manifest files, which can be an expensive operation when the table is large

Re: REST Catalog based Integration Test for Query Engines

2024-09-23 Thread Yufei Gu
+1 for using the REST catalog in the tests. Thanks Haizhou for doing this! Yufei On Thu, Sep 19, 2024 at 12:41 AM Eduard Tudenhöfner < etudenhoef...@apache.org> wrote: > Thanks for looking into this Haizhou. I'll take a closer look at the PRs > this/next week. > > Eduard > > On Thu, Sep 19, 202

Re: [VOTE] Drop Python3.8 Support in PyIceberg 0.8.0

2024-09-23 Thread Daniel Weeks
+1 On Mon, Sep 23, 2024 at 1:48 PM Ameena Ansari wrote: > +1 > > On Mon, 23 Sept 2024 at 13:33, rdb...@gmail.com wrote: > >> +1 >> >> On Mon, Sep 23, 2024 at 10:31 AM Steven Wu wrote: >> >>> +1 (binding). makes sense. >>> >>> On Mon, Sep 23, 2024 at 9:38 AM Yufei Gu wrote: >>> +1 Thanks

Re: [VOTE] Drop Python3.8 Support in PyIceberg 0.8.0

2024-09-23 Thread Ameena Ansari
+1 On Mon, 23 Sept 2024 at 13:33, rdb...@gmail.com wrote: > +1 > > On Mon, Sep 23, 2024 at 10:31 AM Steven Wu wrote: > >> +1 (binding). makes sense. >> >> On Mon, Sep 23, 2024 at 9:38 AM Yufei Gu wrote: >> >>> +1 Thanks for bringing this up. >>> >>> Yufei >>> >>> >>> On Mon, Sep 23, 2024 at 9:

Re: [VOTE] Drop Python3.8 Support in PyIceberg 0.8.0

2024-09-23 Thread rdb...@gmail.com
+1 On Mon, Sep 23, 2024 at 10:31 AM Steven Wu wrote: > +1 (binding). makes sense. > > On Mon, Sep 23, 2024 at 9:38 AM Yufei Gu wrote: > >> +1 Thanks for bringing this up. >> >> Yufei >> >> >> On Mon, Sep 23, 2024 at 9:27 AM Kevin Liu wrote: >> >>> +1 non-binding. Thanks for starting this conve

Re: [VOTE] Drop Python3.8 Support in PyIceberg 0.8.0

2024-09-23 Thread Steven Wu
+1 (binding). makes sense. On Mon, Sep 23, 2024 at 9:38 AM Yufei Gu wrote: > +1 Thanks for bringing this up. > > Yufei > > > On Mon, Sep 23, 2024 at 9:27 AM Kevin Liu wrote: > >> +1 non-binding. Thanks for starting this conversation! >> >> >> On Fri, Sep 20, 2024 at 2:02 PM Sung Yun wrote: >>

Re: [VOTE] Drop Python3.8 Support in PyIceberg 0.8.0

2024-09-23 Thread Yufei Gu
+1 Thanks for bringing this up. Yufei On Mon, Sep 23, 2024 at 9:27 AM Kevin Liu wrote: > +1 non-binding. Thanks for starting this conversation! > > > On Fri, Sep 20, 2024 at 2:02 PM Sung Yun wrote: > >> Hi folks, >> >> I'd like to start this thread to vote on dropping the support for >> Pytho

Re: [VOTE] Drop Python3.8 Support in PyIceberg 0.8.0

2024-09-23 Thread Kevin Liu
+1 non-binding. Thanks for starting this conversation! On Fri, Sep 20, 2024 at 2:02 PM Sung Yun wrote: > Hi folks, > > I'd like to start this thread to vote on dropping the support for > Python3.8 in the upcoming 0.8.0 PyIceberg release. > > Python3.8 will be End-Of-Life in October 2024, and so