ything special for the akka
configurations.
From: Joshua Griffith [mailto:jgriff...@campuslabs.com]
Sent: Thursday, November 16, 2017 2:57 PM
To: Chan, Regina [Tech]
Cc: user@flink.apache.org<mailto:user@flink.apache.org>
Subject: Re: Job Manager Configuration
I have an IO-dominated batch
To: Chan, Regina [Tech]
Cc: user@flink.apache.org
Subject: Re: Job Manager Configuration
I have an IO-dominated batch job with 471 distinct tasks (3786 tasks with
parallelism) running on 8 nodes with 12 GiB of memory and 4 CPUs each. I
haven’t had any problems adding additional tasks except for 1
I have an IO-dominated batch job with 471 distinct tasks (3786 tasks with
parallelism) running on 8 nodes with 12 GiB of memory and 4 CPUs each. I
haven’t had any problems adding additional tasks except for 1) tasks timing out
the first time the cluster is started (I suppose the JVM needs to war
; across the TaskManagers.
>
>
>
> I’ll get you the logs shortly.
>
>
>
> *From:* Till Rohrmann [mailto:trohrm...@apache.org]
> *Sent:* Wednesday, November 08, 2017 10:17 AM
> *To:* Chan, Regina [Tech]
> *Cc:* Chesnay Schepler; user@flink.apache.org
>
> *Subjec
euing up and running when it has the
resources to.
From: Chesnay Schepler [mailto:ches...@apache.org<mailto:ches...@apache.org>]
Sent: Wednesday, November 01, 2017 7:09 AM
To: user@flink.apache.org<mailto:user@flink.apache.org>
Subject: Re: Job Manager Configuration
AFAIK there is no t
? I only gave it 10
>> TaskManagers with 2 slots. I’m perfectly fine with it queuing up and
>> running when it has the resources to.
>>
>>
>>
>>
>>
>>
>>
>> *From:* Chesnay Schepler [mailto:ches...@apache.org]
>> *Sent:* Wednesday, November 01,
ve it 10
> TaskManagers with 2 slots. I’m perfectly fine with it queuing up and
> running when it has the resources to.
>
>
>
>
>
>
>
> *From:* Chesnay Schepler [mailto:ches...@apache.org]
> *Sent:* Wednesday, November 01, 2017 7:09 AM
> *To:* user@flink.apache.o
e.org
Subject: Re: Job Manager Configuration
AFAIK there is no theoretical limit on the size of the plan, it just depends on
the available resources.
The job submissions times out since it takes too long to deploy all the
operators that the job defines. With 300 flows, each with 6 operators y
The user code for all the flows is common though so is there an inefficiency
here in terms of copying this code for every operator?
From: Chesnay Schepler [mailto:ches...@apache.org]
Sent: Wednesday, November 01, 2017 7:09 AM
To: user@flink.apache.org
Subject: Re: Job Manager Configuration
AFAIK there is no theoretical limit on the size of the plan, it just
depends on the available resources.
The job submissions times out since it takes too long to deploy all the
operators that the job defines. With 300 flows, each with 6 operators
you're looking at potentially (1800 * paralleli
Asking an additional question, what is the largest plan that the JobManager can
handle? Is there a limit? My flows don't need to run in parallel and can run
independently. I wanted them to run in one single job because it's part of one
logical commit on my side.
Thanks,
Regina
From: Chan, Regi
11 matches
Mail list logo
