>> So does this issue relate to transactions which are used only when >> exactly_once guarantee is set?
Correct. On 5/8/20 6:28 AM, Pushkar Deole wrote: > Hello Matthias, > > By the way, this error seems to be occurring in only one of the services. > There is another service which is also using kafka streams to consumer from > source, uses processors and then a sink to the output topic, however that > service is running fine. The difference is this other service is using > at_least_once guarantee while the service in error is exactly once > guarantee. > So does this issue relate to transactions which are used only when > exactly_once guarantee is set? > > On Mon, Apr 27, 2020 at 12:37 PM Pushkar Deole <pdeole2...@gmail.com> wrote: > >> came across this: seems to be the one >> https://issues.apache.org/jira/browse/KAFKA-8710 >> >> On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole <pdeole2...@gmail.com> >> wrote: >> >>> Thanks... can you point to those improvements/bugs that are fixed in 2.5? >>> >>> On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax <mj...@apache.org> wrote: >>> >>>> Well, what you say is correct. However, it's a "bug" in the sense that >>>> for some cases the producer does not need to fail, but can re-initialize >>>> itself automatically. Of course, you can also see this as an improvement >>>> and not a bug :) >>>> >>>> >>>> -Matthias >>>> >>>> On 4/25/20 7:48 AM, Pushkar Deole wrote: >>>>> version used is 2.3 >>>>> however, not sure if this is a bug.. after doing some search, came >>>> across >>>>> following for the reason of this: >>>>> >>>>> essentially, the transaction coordinator of streams is cleaning up the >>>>> producer and transaction ids after a certain time interval controller >>>> by >>>>> transactional.id.expiration.ms >>>>> < >>>> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms >>>>> , >>>>> if the coordinator doesn't receive any updates/writes from the >>>> producer for >>>>> that much time. Default of this parameter is 7 days and our labs have >>>> been >>>>> idle for more than that. >>>>> >>>>> On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax <mj...@apache.org> >>>> wrote: >>>>> >>>>>> This version are you using? >>>>>> >>>>>> Couple of broker and client side exactly-once related bugs got fix in >>>>>> the latest release 2.5.0. >>>>>> >>>>>> >>>>>> -Matthias >>>>>> >>>>>> On 4/23/20 11:59 PM, Pushkar Deole wrote: >>>>>>> Hello All, >>>>>>> >>>>>>> While using kafka streams application, we are intermittently getting >>>>>>> following exception and stream is closed. We need to restart the >>>>>>> application to get it working again and start processing. This >>>> exception >>>>>> is >>>>>>> observed in some of the labs which are being idle for some time but >>>> it is >>>>>>> not observed always. Any inputs appreciated here. >>>>>>> >>>>>>> >>>>>> >>>> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread >>>>>>> [analytics-event-filter-StreamThread-1] Failed to commit stream task >>>> 2_14 >>>>>>> due to the following >>>>>>> >>>>>> >>>> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":40000,"stack_trace":"org.apache.kafka.common.KafkaException: >>>>>>> Unexpected error in AddOffsetsToTxnResponse: The producer attempted >>>> to >>>>>> use >>>>>>> a producer id which is not currently assigned to its transactional >>>>>>> id.\n\tat >>>>>>> >>>>>> >>>> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat >>>>>>> >>>>>> >>>> org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat >>>>>>> >>>>>> >>>> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat >>>>>>> >>>>>> >>>> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat >>>>>>> >>>> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat >>>>>>> >>>>>> >>>> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat >>>>>>> >>>>>> >>>> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat >>>>>>> >>>>>> >>>> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat >>>>>>> java.base/java.lang.Thread.run(Unknown Source)\n"} >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >
signature.asc
Description: OpenPGP digital signature
