Just for the record, this conversation continued and resolved in user list [1] and in the PR [2].
[1] https://lists.apache.org/thread/h5mv6ld4l2g4hsjszfdos9f365nh7ctf [2] https://github.com/apache/flink/pull/25509 G On Tue, Oct 15, 2024 at 4:26 PM Gabor Somogyi <gabor.g.somo...@gmail.com> wrote: > Hi All, > > As a result of our investigation we've found major performance degradation > described here [1]. > > TLDR: list operator state restore from S3 is significantly slower from > 1.18.2. Both uncompressed > and compressed state is effected. ~6Mb restore takes ~62 hours. > > We've created already a solution for uncompressed state [2]. This works > fine under heavy load, > but important to mention here that the compressed state still suffers from > this issue. > > Since I was not there when the compressed state was born I would ask some > help to sort that part > out too. > > [1] https://issues.apache.org/jira/browse/FLINK-36530 > [2] https://github.com/apache/flink/pull/25509 > > BR, > G > >
