Hi Florian,
you can rate-limit the Kafka consumer by implementing a custom
DeserializationSchema that sleeps a bit from time to time (or at each
deserialization step)
On Tue, Jan 24, 2017 at 1:16 PM, Florian König
wrote:
> Hi Till,
>
> thank you for the very helpful hints. You are right, I alre
Hi Till,
thank you for the very helpful hints. You are right, I already see
backpressure. In my case, that’s ok because it throttles the Kafka source.
Speaking of which: You mentioned putting the rate limiting mechanism into the
source. How can I do this with a Kafka source? Just extend the Pro
Hi Florian,
any blocking of the user code thread is in general a not so good idea
because the checkpointing happens under the very same lock which also
guards the user code invocation. Thus any checkpoint barrier arriving at
the operator has only the chance to trigger the checkpointing once the
bl
Hi,
You might find this similar thread from the mailing list archive helpful :
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/throttled-stream-td6138.html
.
Best,
Yassine
2017-01-20 10:53 GMT+01:00 Florian König :
> Hi,
>
> i need to limit the rate of processing in a Flink
