Re: Behavior of dropping table with HadoopCatalog

2023-08-30 Thread Manish Malhotra
Yes, that always make sense On Wed, Aug 30, 2023 at 8:30 AM Ryan Blue wrote: > This is one of the many problems with not using a real catalog. If there > are any metadata files under the table location, then the table exists so > we have to delete. The solution is to not use the HadoopCatalog, w

Re: Scan column metrics

2023-10-10 Thread Manish Malhotra
Thanks Ryan, Good point, that makes sense. Though +1 for the feature. We can avoid it during ingestion as well, though we might need the stats some time later, so having options during reading will help. Thanks, Manish On Mon, Oct 9, 2023 at 4:38 PM Ryan Blue wrote: > For that use case, it s

Re: Scan column metrics

2023-11-10 Thread Manish Malhotra
the additional complexity. > > Please review the PR to see for yourselves :) > > Thanks, > Peter > > Manish Malhotra ezt írta (időpont: 2023. > okt. 10., K, 17:02): > >> Thanks Ryan, >> >> Good point, that makes sense. >> >> Though +1 fo

Re: Materialized view integration with REST spec

2024-02-20 Thread Manish Malhotra
Very excited for MV to be in Iceberg :) Keeping in the same doc. would be helpful, to have the trail. But also agreed, if there are too many directions/threads, then keep closing the old one, if there are no more questions. And put down the assumptions for the initial version to move forward. On

Re: [ANNOUNCE] Iceberg Summit Call for Proposals

2024-03-11 Thread Manish Malhotra
+1 for the lightning talk. Thanks Ryan and team, for starting the Iceberg summit! On Thu, Mar 7, 2024 at 5:06 AM Jean-Baptiste Onofré wrote: > Hi Ajantha, > > We will discuss that within the Selection Committee. > I think it would be great to reserve one slot (30mn) for lightning > talks (as we

Re: Materialized view integration with REST spec

2024-03-24 Thread Manish Malhotra
Thanks Walaa, Option1 seems to be a better one, and one of the primary reason is how to keep it simple for the engine. Regrds, Manish On Sun, Mar 24, 2024 at 5:02 AM Renjie Liu wrote: > Hi, Walaa: > > Thanks for your summary. I lean toward option 1, due to the huge effort > for engines to adop

Re: [DISCUSS] camel-iceberg component

2024-05-21 Thread Manish Malhotra
Is Camel can be used as an alternate to Flink? On Tue, May 21, 2024 at 10:17 AM Ryan Blue wrote: > This is an interesting idea. What is the use case and where should this > live? I'm unfamiliar with Camel and I'm not sure what the normal thing is. > At least in the Iceberg community, we general

Re: [Proposal] REST Spec: Server-side Metadata Tables

2024-07-04 Thread Manish Malhotra
Thanks Yufei! This is interesting, and what in my mind as well, as this is the natural progression of the REST Catalog. Totally agreed on enabling metadata from any platform/language, which is right now typically is from the query engines. Though, I feel, users need sql engines to analyze and deb

Re: [DISCUSS] Implementing a table-level statistics file to store column statistics

2024-08-07 Thread Manish Malhotra
First of all thanks a lot Huaxin for starting an important proposal and thread! A lot of important points are already discussed. For me, my thoughts were also tilting towards the partition level stats, what Piotr, Alex, Anton and a few others have mentioned as well. IMO, partition level stats mi

Re: Welcome new committer and PPMC member Ratandeep Ratti

2020-02-17 Thread Manish Malhotra
Congratulations 🎉!! On Sun, Feb 16, 2020 at 8:37 PM RD wrote: > Thanks everyone! > > -Best, > R. > > On Sun, Feb 16, 2020 at 7:39 PM David Christle > wrote: > >> Congrats!!! >> >> >> >> *From: *Jacques Nadeau >> *Reply-To: *"dev@iceberg.apache.org" >> *Date: *Sunday, February 16, 2020 at 7:20

Re: In Remembrance of Kyle

2022-12-05 Thread Manish Malhotra
Very sad and depressing news!! Difficult to believe for such a smiling and jubilant person. Have not worked directly with him, but have followed some of his threads in Iceberg and Trino channels. I have also interacted briefly while he was at Apple. But knows that he has big contribution to the Ic

Re: Welcome Huaxin Gao as a committer!

2025-02-07 Thread Manish Malhotra
Congratulations Huaxin! On Thu, Feb 6, 2025 at 6:34 PM Ron Liu wrote: > Congrats > > Best, > Ron > > roryqi 于2025年2月7日周五 10:21写道: > >> Congrats! >> >> Anton Okolnychyi 于2025年2月7日周五 09:22写道: >> >>> Thanks for the contributions, Huaxin! Congrats! >>> >>> чт, 6 лют. 2025 р. о 13:55 Daniel Weeks

Re: [VOTE] Update partition stats spec for V3

2025-02-01 Thread Manish Malhotra
+1(nonbinding) On Sat, Feb 1, 2025 at 2:49 AM Russell Spitzer wrote: > +1 > > On Sat, Feb 1, 2025 at 3:01 AM Anton Okolnychyi > wrote: > >> Hi all, >> >> I propose the following updates to our partition stats spec in V3: >> >> - Modify `position_delete_record_count` to include a sum of position

Re: Reminder: Iceberg Catalog Community Sync

2025-01-21 Thread Manish Malhotra
Thanks , Can you please share the meeting link as well? Regards, Manish On Tue, Jan 21, 2025 at 2:23 PM Honah J. wrote: > Hi everyone, > > FYI, the first catalog community sync in 2025 will be on tomorrow, > Wednesday 01/22 at 9AM (US/Pacific). Here is the meeting note/recordings: > > https://

Re: [Discuss][Vote] Spec Change - Add optional field added-rows to Snapshot for Row Lineage

2025-01-16 Thread Manish Malhotra
+1, thanks Russel! this will help other engines as well. Thanks, Manish On Thu, Jan 16, 2025 at 3:15 PM Amogh Jahagirdar <2am...@gmail.com> wrote: > I'm +0. I definitely agree with the premise that we need a spec change to > ensure added rows exist at the snapshot level for row lineage, but I f

Re: [DISCUSS] Consolidate docs under Concepts and Project/Terms

2025-02-16 Thread Manish Malhotra
Sorry Manu, I saw the message late. >From some time, I was also thinking that docs needs to be realigned to make navigation easier. Thanks for doing the doc refactoring. Agreed Steven and Russel’s suggestions. Wanted to discuss one suggestion, Shall we have a section for “future release/plans”? W

Re: [VOTE] Minor simplifications for Geo Spec

2025-03-25 Thread Manish Malhotra
Late to the party :) Thanks Szehon +1 (non-binding) On Sun, Mar 23, 2025 at 5:53 PM Szehon Ho wrote: > Thanks all for voting! > > The vote result is: > > +1: 9 (binding: Renjie, Eduard, Fokko, Yufei, Ryan, Daniel, Amogh, > Russell, Szehon), 9 (non-binding: Jia, Gang, Bryan, Huang-Hsiang, Matt,