Thanks for the reply Dawid. The Flink jobs are deployed in Yarn cluster. I am
seeing the error in Job Manager log for some jobs too frequently. I'm using
Flink 1.4.2. I'm running only Streaming Jobs.
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Hi，
I got the same exception when running in flink cluster. The settings is
below:
flink version: 1.5.4
flink-conf.yaml:
jobmanager.heap.mb: 102400
taskmanager.heap.mb: 102400
taskmanager.numberOfTaskSlots: 40
parallelism.default: 40
I have 5 task manager.
My code just read hbase table data an
Hi,
Could you provide us with some more information? Which version of flink
are you running? In which cluster setup? When does this exception occur?
This exception says that request for status overview (no of
taskmanagers, slots info etc.) failed.
Best,
Dawid
On 31/10/2018 20:05, Anil wrote:
>
getting this error in my job manager too frequently. any help. Thanks!
java.util.concurrent.CompletionException: akka.pattern.AskTimeoutException:
Ask timed out on [Actor[akka://flink/user/jobmanager#1927353472]] after
[1 ms]. Sender[null] sent message of type
"org.apache.flink.runtime.message
t;>>>> are you using the RocksDB state backend already?
>>>>>> Maybe writing the state to disk would actually reduce the pressure on
>>>>>> the GC (but of course it'll also reduce throughput a bit).
>>>>>>
>>>>>> A
;> are you using the RocksDB state backend already?
>>>>> Maybe writing the state to disk would actually reduce the pressure on
>>>>> the GC (but of course it'll also reduce throughput a bit).
>>>>>
>>>>> Are there any known issues wi
state to disk would actually reduce the pressure on
>>>> the GC (but of course it'll also reduce throughput a bit).
>>>>
>>>> Are there any known issues with the network? Maybe the network bursts
>>>> on restart cause the timeouts?
>>>>
also reduce throughput a bit).
>>>
>>> Are there any known issues with the network? Maybe the network bursts on
>>> restart cause the timeouts?
>>>
>>>
>>> On Fri, Aug 25, 2017 at 6:17 PM, Steven Wu wrote:
>>>
>>>> Bowen,
>>
issues with the network? Maybe the network bursts on
>> restart cause the timeouts?
>>
>>
>> On Fri, Aug 25, 2017 at 6:17 PM, Steven Wu wrote:
>>
>>> Bowen,
>>>
>>> Heap size is ~50G. CPU was actually pretty low (like <20%) when high GC
>>
uts?
>
>
> On Fri, Aug 25, 2017 at 6:17 PM, Steven Wu wrote:
>
>> Bowen,
>>
>> Heap size is ~50G. CPU was actually pretty low (like <20%) when high GC
>> pause and akka timeout was happening. So maybe memory allocation and GC
>> wasn't really an i
, Aug 25, 2017 at 6:17 PM, Steven Wu wrote:
> Bowen,
>
> Heap size is ~50G. CPU was actually pretty low (like <20%) when high GC
> pause and akka timeout was happening. So maybe memory allocation and GC
> wasn't really an issue. I also recently learned that JVM can pause for
Bowen,
Heap size is ~50G. CPU was actually pretty low (like <20%) when high GC
pause and akka timeout was happening. So maybe memory allocation and GC
wasn't really an issue. I also recently learned that JVM can pause for
writing to GC log for disk I/O. that is another lead I am pursuing.
ob has very high
> memory allocation in steady state. High GC pause then caused akka timeout,
> which then caused jobmanager to think taksmanager containers are
> unhealthy/dead and kill them. And the cycle repeats...
>
> But I hasn't been able to prove or disprove it yet. When I w
; Till/Chesnay, thanks for the answers. Look like this is a result/symptom
>> of underline stability issue that I am trying to track down.
>>
>> It is Flink 1.2.
>>
>> On Fri, Aug 18, 2017 at 12:24 AM, Chesnay Schepler
>> wrote:
>>
>>> The MetricFetch
t I am trying to track down.
>
> It is Flink 1.2.
>
> On Fri, Aug 18, 2017 at 12:24 AM, Chesnay Schepler
> wrote:
>
>> The MetricFetcher always use the default akka timeout value.
>>
>>
>> On 18.08.2017 09:07, Till Rohrmann wrote:
>>
>> Hi Ste
Till/Chesnay, thanks for the answers. Look like this is a result/symptom of
underline stability issue that I am trying to track down.
It is Flink 1.2.
On Fri, Aug 18, 2017 at 12:24 AM, Chesnay Schepler
wrote:
> The MetricFetcher always use the default akka timeout value.
>
>
> On 1
The MetricFetcher always use the default akka timeout value.
On 18.08.2017 09:07, Till Rohrmann wrote:
Hi Steven,
I thought that the MetricFetcher picks up the right timeout from the
configuration. Which version of Flink are you using?
The timeout is not a critical problem for the job
to 60 s in yaml file. I also confirmed the
> setting in Flink UI. But I saw akka timeout of 10 s for metric query
> service. two questions
> 1) why doesn't metric query use the 60 s value configured in yaml file?
> does it always use default 10 s value?
> 2) could this cause
We have set akka.ask.timeout to 60 s in yaml file. I also confirmed the
setting in Flink UI. But I saw akka timeout of 10 s for metric query
service. two questions
1) why doesn't metric query use the 60 s value configured in yaml file?
does it always use default 10 s value?
2) could this
We have set akka.ask.timeout to 60 s in yaml file. I also confirmed the
setting in Flink UI. But I saw akka timeout of 10 s for metric query
service. two questions
1) why doesn't metric query use the 60 s value configured in yaml file?
does it always use default 10 s value?
2) could this
uration file. a value os "200 s" for example increases it to
>>> 200 seconds.
>>>
>>> Greetings,
>>> Stephan
>>>
>>>
>>> On Mon, May 4, 2015 at 11:52 PM, Flavio Pompermaier <
>>> pomperma...@okkam.it> wrote:
>
>
>> As a temporary workaround, you can increase "akka.ask.timeout"in the
>> flink configuration file. a value os "200 s" for example increases it to
>> 200 seconds.
>>
>> Greetings,
>> Stephan
>>
>>
>> On Mon, May 4, 2015 at 11:52
nds.
>
> Greetings,
> Stephan
>
>
> On Mon, May 4, 2015 at 11:52 PM, Flavio Pompermaier
> wrote:
>
>> Hi to all,
>>
>> In my current (local) job I receive a lot of Akka timeout errors during
>> task deploy at:
>>
>>
>> org.apache
the flink
configuration file. a value os "200 s" for example increases it to 200
seconds.
Greetings,
Stephan
On Mon, May 4, 2015 at 11:52 PM, Flavio Pompermaier
wrote:
> Hi to all,
>
> In my current (local) job I receive a lot of Akka timeout e
Hi to all,
In my current (local) job I receive a lot of Akka timeout errors during
task deploy at:
org.apache.flink.runtime.executiongraph.Execution$2.onComplete(Execution.java:342)
is it normal? Which parameter do I have to increase?
Best,
Flavio
25 matches
Mail list logo
