I tested RC2 successfully on EMR with 160 cores across 5 nodes using the performance benchmark [1] with s3 file backend.
I used different backpressure settings to compare aligned and unaligned checkpoints on 1.11-rc2 and aligned checkpoints of 1.10.1. I saw no errors and no regressions (rather we improved the general performance a bit but it's within the margin of error) [2]. I also manually tested failure recovery of unaligned checkpoints by killing taskmanagers under heavy backpressure. [1] https://github.com/dataArtisans/performance [2] https://docs.google.com/spreadsheets/d/18GO15zO-WI2EzK0fTkWucMmAGicmi0mGjxHlHgzitHQ/edit?usp=sharing On Tue, Jun 23, 2020 at 9:41 AM Fabian Paul <fabianp...@data-artisans.com> wrote: > Hi, > > Thanks again for uploading the missing artifacts. Unfortunately this rc > does not fully compile due to [1]. > > Would it be possible for testing purposed to quickly include this fix into > the rc or do you think it is necessary to open a complete new one? > > > [1] https://issues.apache.org/jira/browse/FLINK-18411 < > https://issues.apache.org/jira/browse/FLINK-18411> > > Best, > Fabian -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
