Hi Youzha, do you use exactly once mode in kafka producer? Make sure that you have enabled checkpointing and set the interval appropriately.
You can also see the message flow in the web UI. Check if something is reaching the sink. Aside from that, if you use window operators make sure that they fire. In general, the information that you provided is very sparse. I can give more detailed pointers with more information. On Mon, Nov 30, 2020 at 9:25 AM Youzha <yuza.ras...@gmail.com> wrote: > Hi, i’m using kafka consumer and producer on flink. > > i’ve succeed to consume, transform, and produce to another topic on my > development env ( 1 partition and a few dummy msg) > > but when i try to submit job on production env ( 20 partitions and the > size is about 100Gb ), > there is no one message produce into the kafka topic. i’ve try to set > flink parallelism into 20 but is still not working. then i try to use > rebalance function it’s success to consume the messages but it still no one > msg produce into kafka topic. > > pls advice > -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
