Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1466

[Apache Jenkins Server]

See 

[GitHub] [kafka-site] satishd opened a new pull request, #470: MINOR Add Satish as a committer

[GitBox]

satishd opened a new pull request, #470:
URL: https://github.com/apache/kafka-site/pull/470

   MINOR Add Satish as a committer


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka-site] jlprat commented on pull request #470: MINOR Add Satish as a committer

[GitBox]

jlprat commented on PR #470:
URL: https://github.com/apache/kafka-site/pull/470#issuecomment-1365282565

   Congrats again!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (KAFKA-14553) RecordAccumulator hangs in infinite NOP loop

[Jira]

Viczai Gábor created KAFKA-14553:


 Summary: RecordAccumulator hangs in infinite NOP loop
 Key: KAFKA-14553
 URL: https://issues.apache.org/jira/browse/KAFKA-14553
 Project: Kafka
  Issue Type: Bug
  Components: clients
Affects Versions: 3.3.1
 Environment: - Spring Boot 3.0.1
- Spring Cloud 2022.0.0

Versions of dependencies are defined in boms of SB and SC:

- micrometer-tracing-bridge-brave 1.0.0
- zipkin-reporter-brave 2.16.3
- zipkin-sender-kafka 2.16.3
Reporter: Viczai Gábor


*Summary:*
There is an infinite loop in RecordAccumulator, if stickyBatchSize is 
configured to be 0 in BuiltinPartitioner.
(Which is the default case when using KafkaSender's default Builder.)

*Details:*
The infinite loop is caused by this while:
https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L293
and this continue particularly:
https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L316
because the partitionChanged() call in the condition always return true if 
batchSize is 0.

So program flow never reaches this point:
https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L318
Thus no span data sent to Kafka ever.

The problematic line in partitionChanged() is when it calls an update on the 
BuiltInPartitioner:
https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L242
which in fact always updates the partition because of this condition:
https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/BuiltInPartitioner.java#L218
therefore the next confdition in RecordAccumulator will evaluate to true also:
https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L243
thus returning 'true' and forcing the 'continue' in the while(true) loop.

Suggested fix:
I think these conditions should be changed:
https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/BuiltInPartitioner.java#L218
The equal signs should be removed from the conditions:
{code}if (producedBytes > stickyBatchSize && enableSwitch || producedBytes > 
stickyBatchSize * 2) {{code}
(Btw: line 213 also needs this modification.)

*Note:*
The problem arises because KafkaSender sets the batchSize to 0.
https://github.com/openzipkin/zipkin-reporter-java/blob/2.16.3/kafka/src/main/java/zipkin2/reporter/kafka/KafkaSender.java#L88

*Workaround:*
Simply set the batch size greater than zero.
{code:java}@Configuration
public class SenderConfiguration {

    @Bean
    KafkaSender kafkaSender() {
        Properties overrides = new Properties();
        overrides.put(ProducerConfig.BATCH_SIZE_CONFIG, 1);
        return KafkaSender.newBuilder()
            .bootstrapServers("localhost:9092")
            .topic("zipkin")
            .overrides(overrides)
            .build();
    }
}{code}

*Using:*
- Spring Boot 3.0.1
- Spring Cloud 2022.0.0

pom.xml (fragment):
{code:xml}        
            org.springframework.boot
            spring-boot-autoconfigure
        
        
            org.springframework.boot
            spring-boot-starter-actuator
        
        
            io.micrometer
            micrometer-registry-prometheus
        
        
            io.micrometer
            micrometer-tracing-bridge-brave
        
        
            io.zipkin.reporter2
            zipkin-reporter-brave
        
        
            io.zipkin.reporter2
            zipkin-sender-kafka
        {code}

Everything is on default settings, except a KafkaSender is explicitely created 
as illustrated above. (No autoconfiguration available for Kafka sender.)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (KAFKA-14553) RecordAccumulator hangs in infinite NOP loop

[Luke Chen (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luke Chen resolved KAFKA-14553.
---
Fix Version/s: 3.4.0
   3.3.2
   Resolution: Duplicate

> RecordAccumulator hangs in infinite NOP loop
> 
>
> Key: KAFKA-14553
> URL: https://issues.apache.org/jira/browse/KAFKA-14553
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 3.3.1
> Environment: - Spring Boot 3.0.1
> - Spring Cloud 2022.0.0
> Versions of dependencies are defined in boms of SB and SC:
> - micrometer-tracing-bridge-brave 1.0.0
> - zipkin-reporter-brave 2.16.3
> - zipkin-sender-kafka 2.16.3
>Reporter: Viczai Gábor
>Priority: Minor
> Fix For: 3.4.0, 3.3.2
>
>
> *Summary:*
> There is an infinite loop in RecordAccumulator, if stickyBatchSize is 
> configured to be 0 in BuiltinPartitioner.
> (Which is the default case when using KafkaSender's default Builder.)
> *Details:*
> The infinite loop is caused by this while:
> https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L293
> and this continue particularly:
> https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L316
> because the partitionChanged() call in the condition always return true if 
> batchSize is 0.
> So program flow never reaches this point:
> https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L318
> Thus no span data sent to Kafka ever.
> The problematic line in partitionChanged() is when it calls an update on the 
> BuiltInPartitioner:
> https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L242
> which in fact always updates the partition because of this condition:
> https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/BuiltInPartitioner.java#L218
> therefore the next confdition in RecordAccumulator will evaluate to true also:
> https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java#L243
> thus returning 'true' and forcing the 'continue' in the while(true) loop.
> Suggested fix:
> I think these conditions should be changed:
> https://github.com/apache/kafka/blob/3.3.1/clients/src/main/java/org/apache/kafka/clients/producer/internals/BuiltInPartitioner.java#L218
> The equal signs should be removed from the conditions:
> {code}if (producedBytes > stickyBatchSize && enableSwitch || producedBytes > 
> stickyBatchSize * 2) {{code}
> (Btw: line 213 also needs this modification.)
> *Note:*
> The problem arises because KafkaSender sets the batchSize to 0.
> https://github.com/openzipkin/zipkin-reporter-java/blob/2.16.3/kafka/src/main/java/zipkin2/reporter/kafka/KafkaSender.java#L88
> *Workaround:*
> Simply set the batch size greater than zero.
> {code:java}@Configuration
> public class SenderConfiguration {
>     @Bean
>     KafkaSender kafkaSender() {
>         Properties overrides = new Properties();
>         overrides.put(ProducerConfig.BATCH_SIZE_CONFIG, 1);
>         return KafkaSender.newBuilder()
>             .bootstrapServers("localhost:9092")
>             .topic("zipkin")
>             .overrides(overrides)
>             .build();
>     }
> }{code}
> *Using:*
> - Spring Boot 3.0.1
> - Spring Cloud 2022.0.0
> pom.xml (fragment):
> {code:xml}        
>             org.springframework.boot
>             spring-boot-autoconfigure
>         
>         
>             org.springframework.boot
>             spring-boot-starter-actuator
>         
>         
>             io.micrometer
>             micrometer-registry-prometheus
>         
>         
>             io.micrometer
>             micrometer-tracing-bridge-brave
>         
>         
>             io.zipkin.reporter2
>             zipkin-reporter-brave
>         
>         
>             io.zipkin.reporter2
>             zipkin-sender-kafka
>         {code}
> Everything is on default settings, except a KafkaSender is explicitely 
> created as illustrated above. (No autoconfiguration available for Kafka 
> sender.)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)