I will try to do a ping-pong test this weekend to find out if there are some leaks on pekko side,
On change different from the old transport is , the new one is using pooledByteBufAllocator by default. 何品 PJ Fanning <fannin...@gmail.com> 于2025年1月7日周二 00:42写道： > We don't have any changes in 1.1.3 that might help Flink. > > We haven't yet tracked down a reproducible case that doesn't involve > running the Flink tests. See > https://github.com/apache/pekko/issues/1634 > > On Mon, 6 Jan 2025 at 16:17, Alexander Fedulov > <alexander.fedu...@gmail.com> wrote: > > > > @PJ Fanning, > > Do you have a rough estimate when Pekko 1.1.3 might get released? > > > > Best, > > Alex > > > > > > On Mon, 30 Dec 2024 at 04:12, ConradJam <czy...@apache.org> wrote: > >> > >> By the way, I tried using Flink1.20 to consume Kafka and write Iceberg > >> data, but after using default parameters, it quickly prompted OOM. I > need > >> to increase the taskmanager off heap memory. After improving, I ran it > for > >> 3 days and observed that the Outside JVM Memory of Taskmanager > continued to > >> slowly increase until OOM, which may also be affected by this issue. > >> > >> Matthias Pohl <map...@apache.org> 于2024年12月29日周日 21:22写道： > >> > >> > fyi: The following Flink Jira issues are related to this comment: > >> > - FLINK-36290 [1] OOM in CI > >> > - FLINK-36510 [2]: netty version bump which was backported to 1.20 > and 1.19 > >> > > >> > [1] https://issues.apache.org/jira/browse/FLINK-36290 > >> > [2] https://issues.apache.org/jira/browse/FLINK-36510 > >> > > >> > On Sat, Dec 28, 2024 at 4:02 PM PJ Fanning <fannin...@apache.org> > wrote: > >> > > >> > > It is recommended that you revert back to Pekko 1.0 which uses > Netty 3. > >> > > > >> > > We will notify you when Pekko 1.1.3 is released. > >> > > > >> > > On 2024/12/28 14:54:02 kerr wrote: > >> > > > First, super sorry about this. > >> > > > https://github.com/apache/pekko/pull/1635 > >> > > > > >> > > > I just noticed this issue and did a quick check about it. > >> > > > The memory leak is because I forgot the `Bytebuf#release` call. > >> > > > > >> > > > We will try to work out a release soon on 1.1.3 > >> > > > > >> > > > 何品 > >> > > > > >> > > > >> > > >> > >> > >> -- > >> Best > >> > >> ConradJam >
