Re: When does backpressure matter

2023-06-23 Thread Alexander Fedulov
Hi Lu, I would say that if your application is stable and checkpoints do not timeout there is no immediate necessity to do anything. The fact that the consumer lag stays low means that you are able to keep up with the incoming data. That said, the fact that you observe "constant backpressure" with

When does backpressure matter

2023-06-22 Thread Lu Niu
For example, if a flink job reads from kafka do something and writes to kafka. Do we need to take any actions when the job kafka consumer lag is low or 0 but some tasks have constant backpressure? Do we need to increase the parallelism or do some network tuning so that backpressure is constant 0? I