[jira] [Created] (KAFKA-17969) Fix StorageToolTest for 3.9

Chia-Ping Tsai created KAFKA-17969: -- Summary: Fix StorageToolTest for 3.9 Key: KAFKA-17969 URL: https://issues.apache.org/jira/browse/KAFKA-17969 Project: Kafka Issue Type: Bug R

Re: [VOTE] 3.9.0 RC5

hi Kamal Nice find. I checked those tests, and they are unrelated to production code—they're just something we missed during the backport. I've filed KAFKA-17969 to fix it. Thanks, Chia-Ping On 2024/11/08 10:40:07 Kamal Chandraprakash wrote: > Hi all, > > I took the 3.9 branch

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi Jun, I suggest there are two reasons why ISO8601 is a more appropriate choice than ms. First, kafka-consumer-groups.sh --reset-offsets --by-duration uses ISO8601. This KIP is providing essentially the Kafka consumer equivalent of this tool. I favour consistency with the existing way of doing th

[jira] [Resolved] (KAFKA-17926) Improve the documentation explaining why max.in.flight.requests.per.connection should not exceed 5.

[ https://issues.apache.org/jira/browse/KAFKA-17926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17926. Fix Version/s: 4.0.0 Resolution: Fixed > Improve the documentation explaining why

Re: [VOTE] KIP-1081: Graduation Steps for KIPs

Hi all, I am coming late to this discussion, but I am a bit confused. We seem to talk about levels 1 to 4 (which I think are very opaque and unusual) and at some point we have names: "In Development", "Experimental", "Preview" and "Production Ready". What are we actually proposing? If the latter,

Re: [VOTE] KIP-1081: Graduation Steps for KIPs

Additional feedback: 1. For "Preview", I don't think it has to be complete or api stable and it's definitely not ready for production. My take is that "preview" is close on all of those: roughly speaking no apis changes are planned, the system is robustly tested and the functionality is mostly the

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi, ManiKumar, Thanks for the updated PR. Another comment. JR3. Is there any benefit of using ISO8601 for auto.offset.reset.by.duration instead of just ms in long? The latter covers a broader range. Also, AutoOffsetResetByDuration uses Java During, which covers a different granularity than ISO860

Re: [VOTE] KIP-1081: Graduation Steps for KIPs

Hi Andrew, This is exactly the philosophy behind this KIP. To highlight cases like KIP-932. Best, -- Josep Prat Open Source Engineering Director, Aiven josep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskar

Re: [VOTE] KIP-1102: Enable clients to rebootstrap based on timeout or error code

Thanks everyone for the feedback and votes! +1 (binding) from me The vote has passed with 4 binding votes (Manikumar, Jun, David J, Rajini) and 2 non-binding votes (Andrew, Apoorv). Regards, Rajini On Fri, Nov 8, 2024 at 7:21 AM David Jacot wrote: > +1 (binding). Thanks for the KIP! > > On T

[jira] [Resolved] (KAFKA-17969) Fix StorageToolTest for 3.9

[ https://issues.apache.org/jira/browse/KAFKA-17969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17969. Resolution: Fixed > Fix StorageToolTest for 3.9 > --- > >

[jira] [Resolved] (KAFKA-12894) KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in 3.0)

[ https://issues.apache.org/jira/browse/KAFKA-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-12894. Resolution: Fixed > KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in 3.0) > ---

[jira] [Created] (KAFKA-17974) Upgrade gradle from 8.10 to 8.10.2

Chia-Ping Tsai created KAFKA-17974: -- Summary: Upgrade gradle from 8.10 to 8.10.2 Key: KAFKA-17974 URL: https://issues.apache.org/jira/browse/KAFKA-17974 Project: Kafka Issue Type: Improvemen

[jira] [Resolved] (KAFKA-17570) Rewrite StressTestLog by JMH

[ https://issues.apache.org/jira/browse/KAFKA-17570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17570. Fix Version/s: 4.0.0 Resolution: Fixed > Rewrite StressTestLog by JMH > ---

[jira] [Created] (KAFKA-17973) Relax Restriction for Voters Set Change

Hailey Ni created KAFKA-17973: - Summary: Relax Restriction for Voters Set Change Key: KAFKA-17973 URL: https://issues.apache.org/jira/browse/KAFKA-17973 Project: Kafka Issue Type: Task

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Thanks for the KIP, this is useful. A comment below. On Thu, Nov 7, 2024 at 4:51 PM Matthias J. Sax wrote: > I am personally not convinced that adding a new config > `auto.offset.reset.by.duration` is the best way. Kafka in general has > way too many configs and trying to avoid adding more confi

Re: [VOTE] KIP-1081: Graduation Steps for KIPs

Hi Colin, I think there is perhaps a difference for the very largest KIPs. "In development" means that there's some code merged but there's nothing useful for people except those doing the development. "Experimental" means that there's enough useful code merged that interested people can experim

Re: [VOTE] KIP-1104: Allow Foreign Key Extraction from Both Key and Value in KTable Joins

Hi, Thanks for the KIP, this will be a useful addition. +1(binding) Thanks, Bill On Thu, Nov 7, 2024 at 3:47 PM Matthias J. Sax wrote: > +1 (binding) > > On 11/7/24 3:51 AM, Lucas Brutschy wrote: > > Hi, > > > > thanks for the KIP! > > > > +1 (binding) > > > > Cheers, > > Lucas > > > > On Th

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

I was assuming we will be using the underlying consumer config here. I will update based on your suggestions. Well, yes we can, be for KS we only use the underlying consumer config if there is not code base overwrite. The consumer config only serves as the default. It works similar to `default

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi Matthias, Thanks for the explanation. > I am wondering if AutoOffsetReset should be an interface (we usually just use classes)? I have converted `AutoOffsetReset` to a class now. Thanks. On Fri, Nov 8, 2024 at 11:41 PM Matthias J. Sax wrote: > > >> I was assuming we will be using the unde

[jira] [Created] (KAFKA-17971) Kafka consumer performance has been reduced drastically when added RTT of 50ms to Kafka cluster

Pratheesh M created KAFKA-17971: --- Summary: Kafka consumer performance has been reduced drastically when added RTT of 50ms to Kafka cluster Key: KAFKA-17971 URL: https://issues.apache.org/jira/browse/KAFKA-17971

[jira] [Created] (KAFKA-17970) Move some purgatory classes from core to share

Apoorv Mittal created KAFKA-17970: - Summary: Move some purgatory classes from core to share Key: KAFKA-17970 URL: https://issues.apache.org/jira/browse/KAFKA-17970 Project: Kafka Issue Type:

[jira] [Resolved] (KAFKA-12953) Bump Zookeeper version to 3.6.3

[ https://issues.apache.org/jira/browse/KAFKA-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-12953. Resolution: Won't Fix zk will be removed in kafka 4.x > Bump Zookeeper version to 3.6.3 >

Re: [ANNOUNCE] Apache Kafka 3.9.0

Thanks Colin for all the hard work. Regards, Apoorv Mittal On Fri, Nov 8, 2024 at 6:37 AM Josep Prat wrote: > Hi Colin, > > Thanks for running the release!! > > Best, > > -- > Josep Prat > Open Source Engineering Director, Aiven > josep.p...@aiven.io | +491715557497 | aiven

Re: [VOTE] 3.9.0 RC5

Hi all, I took the 3.9 branch from upstream and ran all the unit tests. The StorageToolTest unit test is failing consistently: % ./gradlew core:test --tests StorageToolTest Gradle Test Run :core:test > Gradle Test Executor 20 > StorageToolTest > testFo

[jira] [Created] (KAFKA-17966) Controller disk replacement does not support scaling up before scaling down

Federico Valeri created KAFKA-17966: --- Summary: Controller disk replacement does not support scaling up before scaling down Key: KAFKA-17966 URL: https://issues.apache.org/jira/browse/KAFKA-17966 Pro

[jira] [Resolved] (KAFKA-17890) Move DelayedOperationPurgatory to server-common

[ https://issues.apache.org/jira/browse/KAFKA-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-17890. Fix Version/s: 4.0.0 Resolution: Fixed > Move DelayedOperationPurgatory to server-c

Re: [VOTE] KIP-1105: Make remote log manager thread-pool configs dynamic

Heya, I have ran a few times into issues which required adjustment of the thread pools and having to carry out a rolling restart was cumbersome. +1 (binding) for the change! Best, Christo On Thu, 7 Nov 2024 at 16:17, Satish Duggana wrote: > +1 (binding) > > On Thu, 7 Nov 2024 at 13:55, Federi

Re: [VOTE] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

Heya, Apologies for the delay. You have my +1 vote as well! I am excited to see this making it in 4.0 Best, Christo On Wed, 6 Nov 2024 at 09:43, Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi all, > > The KIP is passed with 3 (+1) binding votes. Thanks everyone! > > Binding

Re: [ANNOUNCE] Apache Kafka 3.9.0

Thanks for managing the release, Colin! Best, Bruno On 08.11.24 10:49, Apoorv Mittal wrote: Thanks Colin for all the hard work. Regards, Apoorv Mittal On Fri, Nov 8, 2024 at 6:37 AM Josep Prat wrote: Hi Colin, Thanks for running the release!! Best, -- Josep Prat Open So

Re: [VOTE] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

+1 (non binding) from me as well. Thanks. On Fri, Nov 8, 2024 at 10:42 AM Christo Lolov wrote: > > Heya, > > Apologies for the delay. You have my +1 vote as well! I am excited to see > this making it in 4.0 > > Best, > Christo > > On Wed, 6 Nov 2024 at 09:43, Kamal Chandraprakash < > kamal.chand

[jira] [Created] (KAFKA-17967) Dynamic change ProducerStateEntry#NUM_BATCHES_TO_RETAIN based on max.in.flight.requests.per.connection

PoAn Yang created KAFKA-17967: - Summary: Dynamic change ProducerStateEntry#NUM_BATCHES_TO_RETAIN based on max.in.flight.requests.per.connection Key: KAFKA-17967 URL: https://issues.apache.org/jira/browse/KAFKA-17967

Re: [ANNOUNCE] Apache Kafka 3.9.0

I can't imagine anyone else managing the 3.9.0 release as successfully! Best, Chia-Ping Bruno Cadonna 於 2024年11月8日 週五 下午7:23寫道： > Thanks for managing the release, Colin! > > Best, > Bruno > > On 08.11.24 10:49, Apoorv Mittal wrote: > > Thanks Colin for all the hard work. > > > > Regards, > > A

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi All, I have updated the KIP with recent suggestions. Thanks. On Fri, Nov 8, 2024 at 11:53 AM Manikumar wrote: > > Hi Matthias, > > Thanks for the comments. > > > I am personally not convinced that adding a new config > `auto.offset.reset.by.duration` is the best way. > > I think in this case

[jira] [Created] (KAFKA-17968) Create VerifiableShareConsumer

Andrew Schofield created KAFKA-17968: Summary: Create VerifiableShareConsumer Key: KAFKA-17968 URL: https://issues.apache.org/jira/browse/KAFKA-17968 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-17965) New group coordinator request local time excludes coordinator processing

Sean Quah created KAFKA-17965: - Summary: New group coordinator request local time excludes coordinator processing Key: KAFKA-17965 URL: https://issues.apache.org/jira/browse/KAFKA-17965 Project: Kafka

Re: [VOTE] KIP-1103: Additional metrics for cooperative consumption

+1 (binding) On 2024/11/07 11:14:03 Apoorv Mittal wrote: > Hi All, > I would like to start vote on KIP-1103: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1103%3A+Additional+metrics+for+cooperative+consumption > > Discussion thread: > https://lists.apache.org/thread/cfh2n46918b573vqlqj3

[jira] [Created] (KAFKA-17972) Annotate all tests that still require ZK

Colin McCabe created KAFKA-17972: Summary: Annotate all tests that still require ZK Key: KAFKA-17972 URL: https://issues.apache.org/jira/browse/KAFKA-17972 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi, Andrew, Thanks for the reply. I agree that ISO8601 is more human readable and less error prone to configure. It probably covers all common usage. So, we can keep it as it is. Jun On Fri, Nov 8, 2024 at 10:26 AM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Hi Jun, > I sugge