>> So does this issue relate to transactions which are used only when
>> exactly_once guarantee is set?

Correct.

On 5/8/20 6:28 AM, Pushkar Deole wrote:
> Hello Matthias,
> 
> By the way, this error seems to be occurring in only one of the services.
> There is another service which is also using kafka streams to consumer from
> source, uses processors and then a sink to the output topic, however that
> service is running fine. The difference is this other service is using
> at_least_once guarantee while the service in error is exactly once
> guarantee.
> So does this issue relate to transactions which are used only when
> exactly_once guarantee is set?
> 
> On Mon, Apr 27, 2020 at 12:37 PM Pushkar Deole <pdeole2...@gmail.com> wrote:
> 
>> came across this: seems to be the one
>> https://issues.apache.org/jira/browse/KAFKA-8710
>>
>> On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole <pdeole2...@gmail.com>
>> wrote:
>>
>>> Thanks... can you point to those improvements/bugs that are fixed in 2.5?
>>>
>>> On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax <mj...@apache.org> wrote:
>>>
>>>> Well, what you say is correct. However, it's a "bug" in the sense that
>>>> for some cases the producer does not need to fail, but can re-initialize
>>>> itself automatically. Of course, you can also see this as an improvement
>>>> and not a bug :)
>>>>
>>>>
>>>> -Matthias
>>>>
>>>> On 4/25/20 7:48 AM, Pushkar Deole wrote:
>>>>> version used is 2.3
>>>>> however, not sure if this is a bug.. after doing some search, came
>>>> across
>>>>> following for the reason of this:
>>>>>
>>>>> essentially, the transaction coordinator of streams is cleaning up the
>>>>> producer and transaction ids after a certain time interval controller
>>>> by
>>>>> transactional.id.expiration.ms
>>>>> <
>>>> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
>>>>> ,
>>>>> if the coordinator doesn't receive any updates/writes from the
>>>> producer for
>>>>> that much time. Default of this parameter is 7 days and our labs have
>>>> been
>>>>> idle for more than that.
>>>>>
>>>>> On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax <mj...@apache.org>
>>>> wrote:
>>>>>
>>>>>> This version are you using?
>>>>>>
>>>>>> Couple of broker and client side exactly-once related bugs got fix in
>>>>>> the latest release 2.5.0.
>>>>>>
>>>>>>
>>>>>> -Matthias
>>>>>>
>>>>>> On 4/23/20 11:59 PM, Pushkar Deole wrote:
>>>>>>> Hello All,
>>>>>>>
>>>>>>> While using kafka streams application, we are intermittently getting
>>>>>>> following exception and stream is closed. We need to restart the
>>>>>>> application to get it working again and start processing. This
>>>> exception
>>>>>> is
>>>>>>> observed in some of the labs which are being idle for some time but
>>>> it is
>>>>>>> not observed always. Any inputs appreciated here.
>>>>>>>
>>>>>>>
>>>>>>
>>>> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
>>>>>>> [analytics-event-filter-StreamThread-1] Failed to commit stream task
>>>> 2_14
>>>>>>> due to the following
>>>>>>>
>>>>>>
>>>> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":40000,"stack_trace":"org.apache.kafka.common.KafkaException:
>>>>>>> Unexpected error in AddOffsetsToTxnResponse: The producer attempted
>>>> to
>>>>>> use
>>>>>>> a producer id which is not currently assigned to its transactional
>>>>>>> id.\n\tat
>>>>>>>
>>>>>>
>>>> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
>>>>>>>
>>>>>>
>>>> org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat
>>>>>>>
>>>>>>
>>>> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat
>>>>>>>
>>>>>>
>>>> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat
>>>>>>>
>>>> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat
>>>>>>>
>>>>>>
>>>> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat
>>>>>>>
>>>>>>
>>>> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat
>>>>>>>
>>>>>>
>>>> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat
>>>>>>> java.base/java.lang.Thread.run(Unknown Source)\n"}
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
> 

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to