Yeah I don't think that's the case for my setup either :) I wrote a simple
Flink job that just consumes from Kafka and sinks events/sec rate to
Graphite. That consumes from Kafka several orders of magnitude higher than
the job that also sinks to Elasticsearch. As you said, the downstream back
pres
Ah sorry, I forgot to mention that in the docs.
The way that data is pulled from Kafka is bypassing Flink's task
Thread. The topic is consumed in a separate Thread and the task Thread
is just waiting. That's why you don't see any back pressure for Kafka
sources. I would expect your Kafka source to
The new back pressure docs are great, thanks Ufuk! I'm sure those will help
others as well.
In the Source => A => B => Sink example, if A and B show HIGH back
pressure, should Source also show HIGH? In my case it's a Kafka source and
Elasticsearch sink. I know currently our Kafka can provide data
Hey Zach,
just added some documentation, which will be available in ~ 30 mins
here:
https://ci.apache.org/projects/flink/flink-docs-release-1.0/internals/back_pressure_monitoring.html
If you think that something is missing there, I would appreciate some
feedback. :-)
Back pressure is determined
Hi - I'm trying to identify bottlenecks in my Flink streaming job, and am
curious about the Back Pressure view in the job manager web UI. If there
are already docs for Back Pressure please feel free to just point me to
those. :)
When "Sampling in progress..." is displayed, what exactly is happenin
