Re: Running Kafka from IDE

2024-10-23 Thread Pradyumna K
Thank you. Looks like I too accidentally discovered this approach minutes before your reply. I wasn't aware of the DEBUG_SUSPEND_FLAG though. It is useful. Regards Pradyumna

Re: Running Kafka from IDE

2024-10-23 Thread Andrew Schofield
Hi Pradyumna, Yes, you can do this. What I tend to do is to start my broker from a terminal window with the DEBUG_SUSPEND_FLAG and KAFKA_DEBUG environment variables set. This means the broker suspends as it starts and waits for a connection from a debugger. Then I attach IntelliJ to the running

Re: [VOTE] 3.9.0 RC3

2024-10-23 Thread Justine Olshan
Hey Colin, Thanks for running the release. I checked the keys and scanned the docs I built from source, used kraft quickstart, ran a transactions workload, played around with a few other things including running some of the feature version commands. I noticed that some of the KIP-1022 work that

Re: doc clarification about meesage format

2024-10-23 Thread Xiang Zhang
Thank you Greg for all the knowledge, some follow up questions. Does partitionLeaderEpoch always reflect the latest leader election or an old epoch can be allowed ? If it is the first case, then I agree partitionLeaderEpoch should not be included in CRC computation. But it raises some new question

[jira] [Resolved] (KAFKA-17774) Add capability for max fetch records in share fetch

2024-10-23 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-17774. - Fix Version/s: 4.0.0 Resolution: Fixed merged the PR to trunk > Add capability for max fetch rec

Re: [VOTE] KIP-891: Running multiple versions of Connector plugins

2024-10-23 Thread Mickael Maison
Hi, +1 (binding), thanks for the KIP! Mickael On Tue, Oct 22, 2024 at 10:35 PM Greg Harris wrote: > > Hi all, > > Bumping this thread in advance of the November 20th KIP freeze for 4.0. > > Thanks! > Greg > > On Tue, Oct 8, 2024 at 5:03 AM Snehashis wrote: > > > Thanks Greg, Chris and Ashwin >

[jira] [Created] (KAFKA-17862) [buffer pool] corruption during buffer reuse from the pool

2024-10-23 Thread Bharath Vissapragada (Jira)
Bharath Vissapragada created KAFKA-17862: Summary: [buffer pool] corruption during buffer reuse from the pool Key: KAFKA-17862 URL: https://issues.apache.org/jira/browse/KAFKA-17862 Project: Ka

[jira] [Resolved] (KAFKA-17568) Rewrite TestPurgatoryPerformance by Java

2024-10-23 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17568. Fix Version/s: 4.0.0 Resolution: Fixed > Rewrite TestPurgatoryPerformance by Java >

Re: [VOTE] KIP-1054: Support external schemas in JSONConverter

2024-10-23 Thread Greg Harris
Hi Priyanka, This is definitely an improvement for single-schema Connectors, and simple to configure. I think mutable or multiple schemas are fair to consider out-of-scope, as they are currently served by the existing schema management tooling and would complicate this design. I'm also interested

[jira] [Created] (KAFKA-17861) Serialize with ByteBuffer instead of byte[]

2024-10-23 Thread Sarah Hennenkamp (Jira)
Sarah Hennenkamp created KAFKA-17861: Summary: Serialize with ByteBuffer instead of byte[] Key: KAFKA-17861 URL: https://issues.apache.org/jira/browse/KAFKA-17861 Project: Kafka Issue Typ

Re: doc clarification about meesage format

2024-10-23 Thread Greg Harris
Hi Xiang, Thanks for your question! That sentence is a justification for why the partitionLeaderEpoch field is not included in the CRC. If you mutate fields which are included in a CRC, you need to recompute the CRC value. See [1] for mutating the maxTimestamp. Compare that with [2] for setting t

[jira] [Created] (KAFKA-17860) Remove log4j-appender module

2024-10-23 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-17860: -- Summary: Remove log4j-appender module Key: KAFKA-17860 URL: https://issues.apache.org/jira/browse/KAFKA-17860 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

2024-10-23 Thread Lucas Brutschy
Hi Frank, thanks for the KIP! 1) For consistency, should we do the same for kafka-share-groups.sh, ShareGroupDescription, etc. ? Even if we do not implement it right now if the share group implementation may still be incomplete, it may make sense to include it in the KIP. 2) Why call it CONSUMER

[DISCUSS] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

2024-10-23 Thread Frank Yang
Hi all, I would like to kick off the discussion of KIP-1099. This KIP enhances the kafka-consumer-groups tools to include state which is introduced by KIP-848. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+gro

Re: [VOTE] 3.9.0 RC2

2024-10-23 Thread Anton Agestam
Hi Colin, I understand your perspective that this is a bug in the client, but I disagree that this is something that's established, or inferrable from existing specification and implementation. The behavior of exposing the implicit defaults on the wire protocol really is new. The only scenario th