Re: Delete a specific iceberg PositionDelete file and add new one

2022-09-07 Thread Taher Koitawala
Any ideas on this? On Sun, 4 Sep, 2022, 5:56 pm Taher Koitawala, wrote: > Hi All, > Need your help with deleting a positionDelete file that has been > committed before and writing a new one instead. > > The use case is we produce a Position delete file and commit it > to the ta

Re: Spark driver memory issue when scanning all columns of a iceberg table

2022-09-07 Thread Ryan Blue
Manu, The check that you linked to where stats aren’t dropped is when someone is asking for all columns from a manifest file, not when your data query is requesting all columns. In the case of your query, Spark is not asking for stats columns. They will be used for filtering, but will be dropped b

Re: [VOTE] Release Apache Iceberg 0.14.1 RC3

2022-09-07 Thread Ryan Blue
With 5 +1 votes and no -1 or +0 votes, this passes. Thanks, everyone! I'll publish the release. On Mon, Sep 5, 2022 at 9:19 AM John Zhuge wrote: > +1 (non-binding) > > Verified signature, checksum, and license, > > Built and tested with Java 8 and 11. > > Passed internal integration tests with S

In streaming scenarios, only one piece of data is written to a file

2022-09-07 Thread liuyehan
HI,everyone: Now, when I use Iceberg native Java API to write data to Iceberg table, only one piece of data can be written into a file,The next piece of data doesn't fit into the file. Therefore, I need a streaming writing scene to write one piece of data into a file,I also did not find the corr

Meeting Minutes from 9/7 Iceberg Sync

2022-09-07 Thread Sam Redai
Hi Iceberg Community, Here are the minutes and recording from our Iceberg Sync that took place today on *September 7th, 9am-10am PT*. Always remember, anyone can join the discussion so feel free to share the Iceberg-Sync google group with anyone who is s