date:20230121

[jira] [Resolved] (KAFKA-10637) KafkaProducer: IllegalMonitorStateException

2023-01-21 Thread Ismael Juma (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-10637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismael Juma resolved KAFKA-10637.
-
Resolution: Duplicate

Marking as duplicate of KAFKA-10902 since that has a reference to the JDK bug.

> KafkaProducer: IllegalMonitorStateException 
> 
>
> Key: KAFKA-10637
> URL: https://issues.apache.org/jira/browse/KAFKA-10637
> Project: Kafka
>  Issue Type: Bug
>  Components: producer 
>Affects Versions: 2.5.1
>Reporter: Lefteris Katiforis
>Priority: Major
>
> Kafka producer throws the following exception:
> {code:java}
> {\"log\":\"java.lang.IllegalMonitorStateException: current thread is not 
> owner\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.415014714Z\"}"}
>  java.base/java.lang.Object.wait(Native 
> Method)\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.41502027Z\"}"}
> org.apache.kafka.common.utils.SystemTime.waitObject(SystemTime.java:55)\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.415024923Z\"}"}
> at 
> org.apache.kafka.clients.producer.internals.ProducerMetadata.awaitUpdate(ProducerMetadata.java:119)\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.415029863Z\"}"}
> org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:1029)\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.415034336Z\"}"}
> org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:883)\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.415038722Z\"}"}
> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:862)\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.415042939Z\"}"}
> org.springframework.kafka.core.DefaultKafkaProducerFactory$CloseSafeProducer.send(DefaultKafkaProducerFactory.java:781)\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.415047238Z\"}"}
> org.springframework.kafka.core.KafkaTemplate.doSend(KafkaTemplate.java:562)\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.415051555Z\"}"}
> org.springframework.kafka.core.KafkaTemplate.send(KafkaTemplate.java:369)\\n\",\"stream\":\"stdout\",\"time\":\"2020-10-23T12:48:50.415055882Z\"}"}{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-13663) IllegalMonitorStateException in ProducerMetadata.awaitUpdate

2023-01-21 Thread Ismael Juma (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismael Juma resolved KAFKA-13663.
-
Resolution: Duplicate

Marking as duplicate of KAFKA-10902 since that has a reference to the JDK bug.

> IllegalMonitorStateException in ProducerMetadata.awaitUpdate
> 
>
> Key: KAFKA-13663
> URL: https://issues.apache.org/jira/browse/KAFKA-13663
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 3.0.0
>Reporter: RivenSun
>Assignee: Guozhang Wang
>Priority: Major
>
> Since our service kafka-clients was upgraded to {*}2.8.1{*}, sometimes the 
> producer will throw an exception when sending a message, and once this 
> exception occurs, it will continue and will not recover by itself.
> {code:java}
> java.lang.IllegalMonitorStateException: current thread is not owner
>         at java.base/java.lang.Object.wait(Native Method)
>         at 
> org.apache.kafka.common.utils.SystemTime.waitObject(SystemTime.java:55)
>         at 
> org.apache.kafka.clients.producer.internals.ProducerMetadata.awaitUpdate(ProducerMetadata.java:119)
>         at 
> org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:1047)
>         at 
> org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:906)
>         at 
> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:885)
>  {code}
>  
>  
> The version of kafka-clients used before is {*}2.2.2{*}, and this exception 
> has never occurred
> Comparing the changes in the kafkaClient source code, I found that there is a 
> change, it is very likely that this is a *regression bug.*
> In version 2.2.2, the `Metadata#awaitUpdate` method is called in 
> `KafkaProducer#waitOnMetadata`
> But since version {*}2.3.0{*}, `ProducerMetadata#awaitUpdate` is called in 
> `KafkaProducer#waitOnMetadata`
> The most crucial point is:
> `Object#wait(long timeoutMillis)` method,
> In the `Metadata#awaitUpdate` method, thread safety is ensured by the 
> `synchronized` keyword on the `Metadata#awaitUpdate` method;
> In the `ProducerMetadata#awaitUpdate` method, in addition to the 
> `synchronized` on the `ProducerMetadata#awaitUpdate` method, 
> {color:#FF}*there is a second `synchronized` in the 
> `SystemTime#waitObject` method;*{color}
> Although we all know that `synchronized` is reentrant, it is not clear 
> whether the implementation of `ProducerMetadata#awaitUpdate` is inconsistent 
> in different versions of JDK, resulting in this exception; what is certain is 
> that this exception has never occurred before the 2.2.2 version of 
> KafkaProducer
>  
> h2. Suggestion:
> We can keep only one `synchronized` and remove the `synchronized` on the 
> `ProducerMetadata#awaitUpdate` method



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-13296) Verify old assignment within StreamsPartitionAssignor

2023-01-21 Thread Sagar Rao (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sagar Rao resolved KAFKA-13296.
---
Resolution: Fixed

> Verify old assignment within StreamsPartitionAssignor
> -
>
> Key: KAFKA-13296
> URL: https://issues.apache.org/jira/browse/KAFKA-13296
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Reporter: Matthias J. Sax
>Assignee: Sagar Rao
>Priority: Major
>
> `StreamsPartitionAssignor` is responsible to assign partitions and tasks to 
> all StreamsThreads within an application.
> While it ensures to not assign a single partition/task to two threads, there 
> is limited verification about it. In particular, we had one incident for with 
> a zombie thread/consumer did not cleanup its own internal state correctly due 
> to KAFKA-12983. This unclean zombie-state implied that the _old assignment_ 
> reported to `StreamsPartitionAssignor` contained a single partition for two 
> consumers. As a result, both threads/consumers later revoked the same 
> partition and the zombie-thread could commit it's unclean work (even if it 
> should have been fenced), leading to duplicate output under EOS_v2.
> We should consider to add a check to `StreamsPartitionAssignor` if the _old 
> assignment_ is valid, ie, no partition should be missing and no partition 
> should be assigned to two consumers. For this case, we should log the invalid 
> _old assignment_ and send an error code back to all consumer that indicates 
> that they should shut down "unclean" (ie, without and flushing and no 
> committing any offsets or transactions).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-10652) Raft leader should flush accumulated writes after a min size is reached

2023-01-21 Thread Sagar Rao (Jira)



 [ 
https://issues.apache.org/jira/browse/KAFKA-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sagar Rao resolved KAFKA-10652.
---
Resolution: Won't Fix

Not sure this is needed anymore.

> Raft leader should flush accumulated writes after a min size is reached
> ---
>
> Key: KAFKA-10652
> URL: https://issues.apache.org/jira/browse/KAFKA-10652
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Jason Gustafson
>Assignee: Sagar Rao
>Priority: Major
>
> In KAFKA-10601, we implemented linger semantics similar to the producer to 
> let the leader accumulate a batch of writes before fsyncing them to disk. 
> Currently the fsync is only based on the linger time, but it would be helpful 
> to make it size-based as well. In other words, if we accumulate a 
> configurable N bytes, then we should not wait for linger expiration and 
> should just fsync immediately.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)