ask.java:779)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
at java.lang.Thread.run(Thread.java:748)
rui chen 于2024年10月9日周三 11:38写道：
> We found a deadlock problem when a single piece of data is too large on
> Flink1.13.2, do not continue to process the data, which more understand the
> data trans
We found a deadlock problem when a single piece of data is too large on
Flink1.13.2, do not continue to process the data, which more understand the
data transmission piece, welcome to comment.
-- Forwarded message -
发件人： rui chen
Date: 2024年9月29日周日 10:00
Subject: Flink has a large number of data and appears to be suspended after
restarting.
To:
1.单条数据500kb
2.tm 失败后作业重新启动
We encountered similar problems in production, and we want to integrate
FLINK-24035 to solve them, but we don't know how to repeat the problem.
(1.19).
>
>
> Best,
> Zakelly
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-306%3A+Unified+File+Merging+Mechanism+for+Checkpoints
> [2] https://issues.apache.org/jira/browse/FLINK-32070
>
> On Fri, Oct 13, 2023 at 6:28 PM rui chen wrote:
> >
>
After the task restart of our 1.13 version, kakfa consumption zero problem
occurred. Have you ever encountered it?
We found that for some tasks, the JM memory continued to increase. I set
the parameter of state.backend.fs.memory-threshold to 0, and the JM memory
would no longer increase, but many small files might be written in this
way. Does the community have any optimization plan for this area?
ems that you are trying to restore from a not completed or failed
> checkpoint.
>
> On Thu, Sep 28, 2023 at 6:09 PM rui chen wrote:
>
> > When we use 1.13.2,we have the following error:
> > FileNotFoundException: Cannot find metata file metadats in directory
> > '
When we use 1.13.2,we have the following error:
FileNotFoundException: Cannot find metata file metadats in directory
'hdfs://xx/f408dbe327f9e5053e76d7b5323d6e81/chk-173'.
hi Feng
Are you using the open jdk or the oracle jdk?
Best，
rui
rui chen 于2023年9月27日周三 20:22写道：
> hi Feng，
>
> Thanks for your reply, we are 8 u192 JDK, may be is the question, I found
> a JDK issue:https://bugs.openjdk.org/browse/JDK-8215355.
>
> Best,
> rui
>
> Fe
the JDK version.
>
> If you are using JDK 8, you can try using OpenJDK 8u265 as a possible
> solution.
>
>
> Best,
> Feng
>
>
> On Wed, Sep 27, 2023 at 8:08 PM rui chen wrote:
>
>>
>>
>> rui chen 于2023年9月27日周三 19:32写道：
>>
>>> hi
rui chen 于2023年9月27日周三 19:32写道：
> hi Feng,
>
> Thank you for your reply，We observed the GC situation, there is no change
> before and after replacement, several tasks on our line using jemalloc have
> appeared stuck, after removing jemalloc, no stuck situation has been found.
After using the jemalloc memory allocator for a period of time, checkpoint
timeout occurs and tasks are stuck. Who has encountered this? flink
version:1.13.2, jiemalloc version: 5.3.0
13 matches
Mail list logo
