Hi, I would like to know if back pressure applies to operators in the same operator chain?
The background is that I have a simple streaming job that consumes data from Kafka, do some transformation and writes to HDFS (all the operators are chained together), and if the Kafka partitions are much greater that job parallelism (like 40:1), OOM happens. The the root cause should be Kafka consumer pulling too much data. So I’m wondering if I should separate the source and sink to make the back pressure mechanism working. Best, Paul Lam
