lied to all GET-based endpoints, though this might broaden the scope significantly.Thank you.-Original Message-----From: "Shani Elharrar" To: ;Cc: ;Sent: 2024-11-18 (월) 16:21:16 (UTC+09:00)Subject: Re: [DISCUSS] REST: Way to query if metadata pointer is the latestUsing the metadata file
ards
> JB
>
> On Tue, Nov 12, 2024 at 2:53 PM Fokko Driesprong <fo...@apache.org> wrote:
> >
> > Hey Gabor,
> >
> > Thanks for raising this. While reading this, my first thought is to leverage the `tableExists` operation:
> > https://github.com/apache/iceberg/
I recommend option (b), provided there is no partial metadata loading. We
implemented option (b) internally to facilitate partial metadata loading, as we
have tables with hundreds of thousands of snapshots. This results in metadata
that occupies approximately 500 MB in memory (excluding the Json
ob to track the inverted index internally (which is what I had in mind). It can't be shared/synchronized with other Flink jobs or other engines writing to the same table.Thanks,StevenOn Fri, Nov 1, 2024 at 10:50 AM Shani Elharrar wrote:Even if Flink can create this state, it would have to be
t be shared/synchronized with other Flink jobs or other engines writing to the same table.Thanks,StevenOn Fri, Nov 1, 2024 at 10:50 AM Shani Elharrar wrote:Even if Flink can create this state, it would have to be maintained against the Iceberg table, we wouldn't like duplicates (keys) if other
Even if Flink can create this state, it would have to be maintained against the Iceberg table, we wouldn't like duplicates (keys) if other systems / users update the table (e.g manual insert / updates using DML). Shani.On 1 Nov 2024, at 18:32, Steven Wu wrote:> Add support for inverted indexes to
+1 for option (a). Shani.On 21 Aug 2024, at 19:07, Péter Váry wrote:I think from the correctness perspective only the option (a) is valid. The difference between snapshot2 and snapshot3 is one delete and one insertion.Jason Fine ezt írta (időpont: 2024. aug. 21., Sze, 15:26):Great to see someone
