Well, it's for sure EOS related, but it seems to be a different root cause.
I am not aware of any related bug.
-Matthias
On 9/9/20 4:29 AM, Pushkar Deole wrote:
> Hi Matthias,
>
> We are using confluent kafka and upgraded to confluent version 5.5.0 which
> I believe maps to 2.5.0 of apache kafka. We tested on few labs by keeping
> the solution idle for few days and didn't observe the issue.
>
> However on one of the labs we observed issue again recently, this is the
> exception: unfortunately, don't have complete stack trace.
> Anyway, do you think it is same exception as above or is it different? and
> whether this is also a kafka server issue that is being reported already?
>
> {"@timestamp":"2020-08-12T09:17:35.270+00:00","@version":"1","message":"Unexpected
> exception in stream
> processing.","logger_name":"com.avaya.analytics.filter.ApplicationConfig","thread_name":"analytics-event-filter-Str
> eamThread-1","level":"ERROR","level_value":40000,"stack_trace":"org.apache.kafka.common.errors.InvalidPidMappingException:
> The producer attempted to use a producer id which is not currently assigned
> to its transactiona l id.\n"}
>
> On Sat, May 9, 2020 at 12:49 AM Matthias J. Sax <mj...@apache.org> wrote:
>
>>>> So does this issue relate to transactions which are used only when
>>>> exactly_once guarantee is set?
>>
>> Correct.
>>
>> On 5/8/20 6:28 AM, Pushkar Deole wrote:
>>> Hello Matthias,
>>>
>>> By the way, this error seems to be occurring in only one of the services.
>>> There is another service which is also using kafka streams to consumer
>> from
>>> source, uses processors and then a sink to the output topic, however that
>>> service is running fine. The difference is this other service is using
>>> at_least_once guarantee while the service in error is exactly once
>>> guarantee.
>>> So does this issue relate to transactions which are used only when
>>> exactly_once guarantee is set?
>>>
>>> On Mon, Apr 27, 2020 at 12:37 PM Pushkar Deole <pdeole2...@gmail.com>
>> wrote:
>>>
>>>> came across this: seems to be the one
>>>> https://issues.apache.org/jira/browse/KAFKA-8710
>>>>
>>>> On Mon, Apr 27, 2020 at 12:17 PM Pushkar Deole <pdeole2...@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks... can you point to those improvements/bugs that are fixed in
>> 2.5?
>>>>>
>>>>> On Mon, Apr 27, 2020 at 1:03 AM Matthias J. Sax <mj...@apache.org>
>> wrote:
>>>>>
>>>>>> Well, what you say is correct. However, it's a "bug" in the sense that
>>>>>> for some cases the producer does not need to fail, but can
>> re-initialize
>>>>>> itself automatically. Of course, you can also see this as an
>> improvement
>>>>>> and not a bug :)
>>>>>>
>>>>>>
>>>>>> -Matthias
>>>>>>
>>>>>> On 4/25/20 7:48 AM, Pushkar Deole wrote:
>>>>>>> version used is 2.3
>>>>>>> however, not sure if this is a bug.. after doing some search, came
>>>>>> across
>>>>>>> following for the reason of this:
>>>>>>>
>>>>>>> essentially, the transaction coordinator of streams is cleaning up
>> the
>>>>>>> producer and transaction ids after a certain time interval controller
>>>>>> by
>>>>>>> transactional.id.expiration.ms
>>>>>>> <
>>>>>>
>> https://docs.confluent.io/current/installation/configuration/broker-configs.html#transactional-id-expiration-ms
>>>>>>> ,
>>>>>>> if the coordinator doesn't receive any updates/writes from the
>>>>>> producer for
>>>>>>> that much time. Default of this parameter is 7 days and our labs have
>>>>>> been
>>>>>>> idle for more than that.
>>>>>>>
>>>>>>> On Fri, Apr 24, 2020 at 10:46 PM Matthias J. Sax <mj...@apache.org>
>>>>>> wrote:
>>>>>>>
>>>>>>>> This version are you using?
>>>>>>>>
>>>>>>>> Couple of broker and client side exactly-once related bugs got fix
>> in
>>>>>>>> the latest release 2.5.0.
>>>>>>>>
>>>>>>>>
>>>>>>>> -Matthias
>>>>>>>>
>>>>>>>> On 4/23/20 11:59 PM, Pushkar Deole wrote:
>>>>>>>>> Hello All,
>>>>>>>>>
>>>>>>>>> While using kafka streams application, we are intermittently
>> getting
>>>>>>>>> following exception and stream is closed. We need to restart the
>>>>>>>>> application to get it working again and start processing. This
>>>>>> exception
>>>>>>>> is
>>>>>>>>> observed in some of the labs which are being idle for some time but
>>>>>> it is
>>>>>>>>> not observed always. Any inputs appreciated here.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>> {"@timestamp":"2020-04-15T13:53:52.698+00:00","@version":"1","message":"stream-thread
>>>>>>>>> [analytics-event-filter-StreamThread-1] Failed to commit stream
>> task
>>>>>> 2_14
>>>>>>>>> due to the following
>>>>>>>>>
>>>>>>>>
>>>>>>
>> error:","logger_name":"org.apache.kafka.streams.processor.internals.AssignedStreamsTasks","thread_name":"analytics-event-filter-StreamThread-1","level":"ERROR","level_value":40000,"stack_trace":"org.apache.kafka.common.KafkaException:
>>>>>>>>> Unexpected error in AddOffsetsToTxnResponse: The producer attempted
>>>>>> to
>>>>>>>> use
>>>>>>>>> a producer id which is not currently assigned to its transactional
>>>>>>>>> id.\n\tat
>>>>>>>>>
>>>>>>>>
>>>>>>
>> org.apache.kafka.clients.producer.internals.TransactionManager$AddOffsetsToTxnHandler.handleResponse(TransactionManager.java:1406)\n\tat
>>>>>>>>>
>>>>>>>>
>>>>>>
>> org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)\n\tat
>>>>>>>>>
>>>>>>>>
>>>>>>
>> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)\n\tat
>>>>>>>>>
>>>>>>>>
>>>>>>
>> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)\n\tat
>>>>>>>>>
>>>>>>
>> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)\n\tat
>>>>>>>>>
>>>>>>>>
>>>>>>
>> org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:425)\n\tat
>>>>>>>>>
>>>>>>>>
>>>>>>
>> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:311)\n\tat
>>>>>>>>>
>>>>>>>>
>>>>>>
>> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat
>>>>>>>>> java.base/java.lang.Thread.run(Unknown Source)\n"}
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>
>>
>>
>
signature.asc
Description: OpenPGP digital signature
