Hello Praneeth, that looks correct then. In that case, maybe somebody else can chime in. Are you aware of this post on SO: https://stackoverflow.com/questions/45047876/apache-kafka-exactly-once-implementation-not-sending-messages
Frank On Sat, Apr 9, 2022 at 7:42 PM Praneeth Ramesh <sr.prane...@gmail.com> wrote: > Hi Frank > > Thanks for the response. > I see that the min.isr value is 2 and replication factor is 4 in my case. > Do you see any issues with these values..? > > Thank you in advance > > > On Fri, Apr 8, 2022 at 11:28 PM Frank Dekervel <fr...@kapernikov.com> > wrote: > >> Hello, >> >> Check if your topic replication factor is not below min.isr setting of >> Kafka. I had the same problem and that was it for me. >> >> Frank >> >> Op za 9 apr. 2022 04:01 schreef Praneeth Ramesh <sr.prane...@gmail.com>: >> >>> Hi All >>> >>> I have a job which reads from kafka and applies some transactions and >>> writes the data back to kafka topics. >>> When I use the exactly once semantics I see that the kafka producer is >>> not initialized and the operator task is hanging in INITIALIZING state and >>> fails eventually. >>> >>> It works fine when I change the mode to AT_LEAST_ONCE. I see that in >>> this mode the transactionId is null in the producer config and >>> enable.idempotence is false. >>> >>> But when I enable EXACTLY_ONCE I see that the producer configs having >>> transactionId which I set and enable.idempotence to true. From trace logs >>> what I can see is the transaction manager is initializing and >>> reinitializing the transactionId again and again and does not make any >>> progress. >>> >>> I tried to analyze the thread dump and see that there are threads >>> blocked at >>> at >>> org.apache.kafka.clients.producer.internals.TransactionalRequestResult.await(TransactionalRequestResult.java:68) >>> at >>> org.apache.kafka.clients.producer.KafkaProducer.initTransactions(KafkaProducer.java:632) >>> at >>> org.apache.flink.connector.kafka.sink.FlinkKafkaInternalProducer.initTransactionId(FlinkKafkaInternalProducer.java:156) >>> >>> >>> I made sure that there are no ACLs problems with brokers. Any idea what >>> could be the problem? >>> >>> Attaching the TM logs where the application is reinitializing the >>> transactions. >>> >>> >>> Thanks for any help or pointers. >>> >>> >>> >>> >>> -- >>> Regards >>> Praneeth Ramesh >>> >> > > -- > Regards > Praneeth Ramesh > -- [image: Kapernikov] <https://kapernikov.com/> Frank Dekervel +32 473 94 34 21 <+32473943421> www.kapernikov.com <https://kapernikov.com/> [image: Blog] <https://www.kapernikov.com/emailsigs/blog/redirector.php>
