t;>>>>>> snapshots will cause an exception by default, and deletes may be
>>>>>>> ignored by
>>>>>>> setting streaming-skip-delete-snapshots=true.
>>>>>>
>>>>>>
>>>>>> https://iceber
what others think.
>>
>> @Gyula
>>
>>> There is a slight risk with 3 that it may break pipelines where the
>>> current behaviour is well understood and expected but that's probably a
>>> very small minority of the users.
>>
>>
>> Agreed
Hi Max!
I like this proposal especially that proper streaming reads of deletes seem
to be quite a bit of work based on recent efforts.
Giving an option to include the append parts of OVERWRITE snapshots (2) is
a great quick improvement that will unblock use-cases where the iceberg
table is used t
he
>> slowness of reading caused by the growing number of equality delete files.
>>
>> Thanks, Amogh, for bringing up the comparison with known alternatives. We
>> should include benchmarks for those as well, to illustrate the trade-offs
>> in read/write performance
Thank you for the proposal!
I agree with what had been said above that we need to narrow down the scope
here and what is the primary target for the optimization.
As Amogh has also pointed out, CDC (streaming) read performance (with
equality deletes) would be one of the biggest beneficiaries of th
