Hi Stephan,
This did not work. For the working case I do see a better utilization of
available slots. However the non working case still doesn't work.
Basically I assigned a unique group to the sources in my for loop - given I
have way more slots than the parallelism I seek.
I know about the par
Hi!
I think what you are seeing is the effect of too mans tasks going to the
same task slot. Have a look here:
https://ci.apache.org/projects/flink/flink-docs-release-1.2/concepts/runtime.html#task-slots-and-resources
By default, Flink shares task slots across all distinct pipelines of the
same p
Actually, I take it back. It is the last union that is causing issues (of job
being un-submittable). If I don’t conbineAtEnd, I can go higher (at least
deploy the job), all the way up to 63.
After that it starts failing in too many files open in Rocks DB (which I can
understand and is at least
Is there a limit on how many DataStreams can be defined in a streaming program?
Looks like flink has problems handling too many data streams? I simplified my
topology further. For eg, this works (parallelism of 4)
However, when I try to go beyond 51 (found empirically by parametrizing
nParts)
I even make it 10 minutes:
akka.client.timeout: 600s
But doesn’t feel like it is taking effect. It still comes out at about the same
time with the same error.
-Abhishek-
> On Jan 23, 2017, at 6:04 AM, Abhishek R. Singh
> wrote:
>
> yes, I had increased it to 5 minutes. It just sits there an
yes, I had increased it to 5 minutes. It just sits there and bails out again.
> On Jan 23, 2017, at 1:47 AM, Jonas wrote:
>
> The exception says that
>
> Did you already try that?
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.co
The exception says that
Did you already try that?
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/weird-client-failure-timeout-tp11201p11204.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
I am using version 1.1.4 (latest stable)
> On Jan 23, 2017, at 12:41 AM, Abhishek R. Singh
> wrote:
>
> I am trying to construct a topology like this (shown for parallelism of 4) -
> basically n parallel windowed processing sub-pipelines with single source and
> single sink:
>
>
>
> I am g
