Re: New kafka producer on each checkpoint

2020-04-13 Thread Becket Qin
gt;>> >>> [1] >>> https://github.com/apache/flink/blob/980e31dcc29ec6cc60ed59569f1f1cb7c47747b7/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer.java#L871 >>> >>> Best >>> Yun Tang >>

Re: New kafka producer on each checkpoint

2020-04-13 Thread Becket Qin
KafkaProducer.java#L871 >> >> Best >> Yun Tang >> ------ >> *From:* Maxim Parkachov >> *Sent:* Monday, April 6, 2020 23:16 >> *To:* user@flink.apache.org >> *Subject:* New kafka producer on each checkpoint >> >> Hi e

Re: New kafka producer on each checkpoint

2020-04-13 Thread Maxim Parkachov
569f1f1cb7c47747b7/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer.java#L871 > > Best > Yun Tang > -- > *From:* Maxim Parkachov > *Sent:* Monday, April 6, 2020 23:16 > *To:*

Re: New kafka producer on each checkpoint

2020-04-07 Thread Yun Tang
/flink/streaming/connectors/kafka/FlinkKafkaProducer.java#L871 Best Yun Tang From: Maxim Parkachov Sent: Monday, April 6, 2020 23:16 To: user@flink.apache.org Subject: New kafka producer on each checkpoint Hi everyone, I'm trying to test exactly once function

New kafka producer on each checkpoint

2020-04-06 Thread Maxim Parkachov
Hi everyone, I'm trying to test exactly once functionality with my job under production load. The job is reading from kafka, using kafka timestamp as event time, aggregates every minute and outputs to other kafka topic. I use checkpoint interval 10 seconds. Everything seems to be working fine, bu