No, I don't think that local sorting in Flink is a good idea, at least for
streaming. I'm not sure what the plan is on the batch side, but it probably
only makes sense for batch.
On Mon, Oct 25, 2021 at 7:42 PM Steven Wu wrote:
> Ryan, thanks a lot for reviewing the proposal thoroughly.
>
> Orig
Another follow-up regarding this : *"Merge strategy that does not do any
bin-packing, and only merges the delete files for each data file and writes
it back. The new data file will have the same sequence number as the old
file before Merge"* ; shouldn't the sequence number be set to the highest
seq
I had a few follow-up points.
1 *"(1) for hot partitions, users can try to only perform Convert and
Rewrite to keep delete file sizes and count manageable, until the partition
becomes cold and a Merge can be performed safely.".* : I believe for the
CDC use case it is hard to guarantee that that p
I closed the upgrade project and marked the FLIP-27 project priority 1.
Thanks for all the work to get this done!
On Sun, Oct 31, 2021 at 8:10 PM OpenInx wrote:
> Update:
>
> I think the project [Flink: Upgrade to 1.13.2][1] in RoadMap can be
> closed now, because all of the issues have been ad
