Re: [PR] KAFKA-15823: disconnect from controller on AuthenticationException [kafka]

2023-11-23 Thread via GitHub
gaurav-narula commented on PR #14760: URL: https://github.com/apache/kafka/pull/14760#issuecomment-1825242721 CC: @hachikuji since you've looked into the response handler recently. Can you please review this PR? -- This is an automated message from the Apache Git Service. To respond to th

[jira] [Commented] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-11-23 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789322#comment-17789322 ] Phuc Hong Tran commented on KAFKA-15341: FYI, I'm writting my findings here so I

[jira] [Commented] (KAFKA-14508) Add ConsumerGroupInstallAssignment API

2023-11-23 Thread Alexander Aghili (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789269#comment-17789269 ] Alexander Aghili commented on KAFKA-14508: -- https://github.com/apache/kafka/pul

[PR] Kafka-14508: ConsumerGroupInstallAssignment API Client [WIP] [kafka]

2023-11-23 Thread via GitHub
Alexander-Aghili opened a new pull request, #14834: URL: https://github.com/apache/kafka/pull/14834 Implemented ConsumerGroupInstallAssignment API on the Client according to KIP-848. Still need to do server implementation (That will be a separate pull request). Also, need to get the proper

[jira] [Comment Edited] (KAFKA-15132) Implement disable & re-enablement for Tiered Storage

2023-11-23 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789243#comment-17789243 ] Jorge Esteban Quilcate Otoya edited comment on KAFKA-15132 at 11/23/23 7:42 PM: --

[jira] [Commented] (KAFKA-15132) Implement disable & re-enablement for Tiered Storage

2023-11-23 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789243#comment-17789243 ] Jorge Esteban Quilcate Otoya commented on KAFKA-15132: -- Just realiz

Re: [PR] MINOR: Update LICENSE-binary for 3.7.0 [kafka]

2023-11-23 Thread via GitHub
mimaison commented on PR #14833: URL: https://github.com/apache/kafka/pull/14833#issuecomment-1824734799 cc @stanislavkozlovski I've have to update `license-BINARY` for 3.6.1 so making the changes for trunk too. -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] MINOR: Update LICENSE-binary for 3.7.0 [kafka]

2023-11-23 Thread via GitHub
mimaison commented on code in PR #14833: URL: https://github.com/apache/kafka/pull/14833#discussion_r1403609727 ## licenses/protobuf-java-BSD-3-clause: ## @@ -0,0 +1,34 @@ +Copyright 2023, the original author or authors. Review Comment: Google does not have a BSD-3-lcause li

[PR] MINOR: Update LICENSE-binary for 3.7.0 [kafka]

2023-11-23 Thread via GitHub
mimaison opened a new pull request, #14833: URL: https://github.com/apache/kafka/pull/14833 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgr

[PR] KAFKA-15883: Implement RemoteCopyLagBytes [kafka]

2023-11-23 Thread via GitHub
clolov opened a new pull request, #14832: URL: https://github.com/apache/kafka/pull/14832 ### Summary This pull request implements the first in the list of metrics in [KIP-963: Additional metrics in Tiered Storage](https://cwiki.apache.org/confluence/display/KAFKA/KIP-963%3A+Additional+m

[jira] [Commented] (KAFKA-15865) Ensure consumer.poll() execute autocommit callback

2023-11-23 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789207#comment-17789207 ] Lucas Brutschy commented on KAFKA-15865: Opened https://github.com/apache/kafka/

[PR] KAFKA-15865: Remove autocommit completion event [kafka]

2023-11-23 Thread via GitHub
lucasbru opened a new pull request, #14831: URL: https://github.com/apache/kafka/pull/14831 There is no callback associated with autocommit, so I do not think we need this event. This closes KAFKA-15865. ### Committer Checklist (excluded from commit message) - [ ] Verify design and

Re: [PR] KIP-978: Allow dynamic reloading of certificates with different DN / SANs [kafka]

2023-11-23 Thread via GitHub
mimaison commented on PR #14756: URL: https://github.com/apache/kafka/pull/14756#issuecomment-1824617646 @viktorsomogyi Do you have any further comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] KAFKA-13531: Disable flaky NamedTopologyIntegrationTest [kafka]

2023-11-23 Thread via GitHub
lucasbru opened a new pull request, #14830: URL: https://github.com/apache/kafka/pull/14830 Named topologies is a feature that is planned to be removed from AK with 4.0 and was never used via the public interface. It was used in a few versions of KSQL only, but was disabled there as well. W

[jira] [Commented] (KAFKA-15865) Ensure consumer.poll() execute autocommit callback

2023-11-23 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789148#comment-17789148 ] Lucas Brutschy commented on KAFKA-15865: Note that `maybeInvokeCommitCallbacks`

[jira] [Commented] (KAFKA-15327) Client consumer should commit offsets on close

2023-11-23 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789147#comment-17789147 ] Lucas Brutschy commented on KAFKA-15327: [~pnee] can this issue be closed? have

[jira] [Commented] (KAFKA-15865) Ensure consumer.poll() execute autocommit callback

2023-11-23 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789139#comment-17789139 ] Lucas Brutschy commented on KAFKA-15865: [~pnee] I just checked this out, but I'

[jira] [Assigned] (KAFKA-15865) Ensure consumer.poll() execute autocommit callback

2023-11-23 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy reassigned KAFKA-15865: -- Assignee: Lucas Brutschy (was: Philip Nee) > Ensure consumer.poll() execute autocomm

[PR] MINOR: Fix flaky test testRackAwareRangeAssignor [kafka]

2023-11-23 Thread via GitHub
ex172000 opened a new pull request, #14829: URL: https://github.com/apache/kafka/pull/14829 The test fails frequently in CI and local dev env. If the offset is not committed, after reassignment the consumer could come back to consume the messages in epoch 0 and cause assertion failure.

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-23 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1403404963 ## core/src/main/scala/kafka/server/ReplicaAlterLogDirsThread.scala: ## @@ -76,13 +83,69 @@ class ReplicaAlterLogDirsThread(name: String, futureLog.updateHighWater

[PR] MINOR: Add retry for CI [kafka]

2023-11-23 Thread via GitHub
ex172000 opened a new pull request, #14828: URL: https://github.com/apache/kafka/pull/14828 There are many flaky tests(see PR14814), adding retry can improve the test robustness. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ]

[jira] [Commented] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2023-11-23 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789127#comment-17789127 ] Lucas Brutschy commented on KAFKA-12679: This is solved in current master and wi

[jira] [Updated] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2023-11-23 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy updated KAFKA-12679: --- Fix Version/s: 3.7.0 > Rebalancing a restoring or running task may cause directory livelocki

[jira] [Assigned] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2023-11-23 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy reassigned KAFKA-12679: -- Assignee: Lucas Brutschy > Rebalancing a restoring or running task may cause director

Re: [PR] KAFKA-15803: Update leader epoch in commitAsync and committed [kafka]

2023-11-23 Thread via GitHub
lucasbru commented on code in PR #14817: URL: https://github.com/apache/kafka/pull/14817#discussion_r140866 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -204,6 +206,24 @@ public void testCommitted() { }

Re: [PR] KAFKA-15445: Add JVM Docker image [kafka]

2023-11-23 Thread via GitHub
VedarthConfluent commented on code in PR #14552: URL: https://github.com/apache/kafka/pull/14552#discussion_r1403318397 ## docker/jvm/Dockerfile: ## @@ -0,0 +1,98 @@ +### +# Licensed to the Apache Softw

Re: [PR] KAFKA-15445: Add JVM Docker image [kafka]

2023-11-23 Thread via GitHub
VedarthConfluent commented on code in PR #14552: URL: https://github.com/apache/kafka/pull/14552#discussion_r1403315403 ## docker/jvm/Dockerfile: ## @@ -0,0 +1,98 @@ +### +# Licensed to the Apache Softw

Re: [PR] KAFKA-15803: Update leader epoch in commitAsync and committed [kafka]

2023-11-23 Thread via GitHub
cadonna commented on code in PR #14817: URL: https://github.com/apache/kafka/pull/14817#discussion_r1403241148 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -204,6 +206,24 @@ public void testCommitted() { }

Re: [PR] KAFKA-13152: Kip 770 buffer size fix [kafka]

2023-11-23 Thread via GitHub
vamossagar12 commented on PR #13283: URL: https://github.com/apache/kafka/pull/13283#issuecomment-1824302840 Thanks @ableegoldman , that makes sense. Well I have another question again due to me not being upto date with the latest developments in streams. I have this test in `StreamThreadTe

[jira] [Resolved] (KAFKA-15840) Correct initialization of ConsumerGroupHeartbeat by client

2023-11-23 Thread Andrew Schofield (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15840. -- Resolution: Duplicate > Correct initialization of ConsumerGroupHeartbeat by client > -

[jira] [Commented] (KAFKA-15840) Correct initialization of ConsumerGroupHeartbeat by client

2023-11-23 Thread Andrew Schofield (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789069#comment-17789069 ] Andrew Schofield commented on KAFKA-15840: -- Fixed by KAFKA-15574. > Correct in

Re: [PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-23 Thread via GitHub
apoorvmittal10 commented on code in PR #14770: URL: https://github.com/apache/kafka/pull/14770#discussion_r1403234553 ## core/src/test/scala/unit/kafka/server/BrokerLifecycleManagerTest.scala: ## @@ -207,34 +207,35 @@ class BrokerLifecycleManagerTest { } @Test - def tes

Re: [PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-23 Thread via GitHub
apoorvmittal10 commented on code in PR #14770: URL: https://github.com/apache/kafka/pull/14770#discussion_r1403234553 ## core/src/test/scala/unit/kafka/server/BrokerLifecycleManagerTest.scala: ## @@ -207,34 +207,35 @@ class BrokerLifecycleManagerTest { } @Test - def tes

Re: [PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-23 Thread via GitHub
apoorvmittal10 commented on code in PR #14770: URL: https://github.com/apache/kafka/pull/14770#discussion_r1403234553 ## core/src/test/scala/unit/kafka/server/BrokerLifecycleManagerTest.scala: ## @@ -207,34 +207,35 @@ class BrokerLifecycleManagerTest { } @Test - def tes

[jira] [Created] (KAFKA-15890) kafka-console-consumer.sh problems with new consumer

2023-11-23 Thread Andrew Schofield (Jira)
Andrew Schofield created KAFKA-15890: Summary: kafka-console-consumer.sh problems with new consumer Key: KAFKA-15890 URL: https://issues.apache.org/jira/browse/KAFKA-15890 Project: Kafka

Re: [PR] MINOR: Correcting Javadoc for ConnectAssertions [kafka]

2023-11-23 Thread via GitHub
vamossagar12 commented on PR #14827: URL: https://github.com/apache/kafka/pull/14827#issuecomment-1824186109 hey @yashmayya , can you review this small PR? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[PR] MINOR: Correcting Javadoc for ConnectAssertions [kafka]

2023-11-23 Thread via GitHub
vamossagar12 opened a new pull request, #14827: URL: https://github.com/apache/kafka/pull/14827 The 2 `checkConnectorState` method implementations have the following text in their return Javadocs: `* @return true if the connector and tasks are in RUNNING state; false otherwise`

[jira] [Resolved] (KAFKA-15856) Add integration tests for JoinGroup API and SyncGroup API

2023-11-23 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-15856. - Fix Version/s: 3.7.0 Resolution: Fixed > Add integration tests for JoinGroup API and Sync

Re: [PR] KAFKA-15856: Add integration tests for JoinGroup API and SyncGroup API [kafka]

2023-11-23 Thread via GitHub
dajac merged PR #14800: URL: https://github.com/apache/kafka/pull/14800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-23 Thread via GitHub
dajac commented on PR #14758: URL: https://github.com/apache/kafka/pull/14758#issuecomment-1824112828 I just merged https://github.com/apache/kafka/pull/14781. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] KAFKA-15756: [1/2] Migrate existing integration tests to run old protocol in new coordinator [kafka]

2023-11-23 Thread via GitHub
dajac merged PR #14781: URL: https://github.com/apache/kafka/pull/14781 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15888: DistributedHerder log context should not use the same client ID for each Connect worker by default [kafka]

2023-11-23 Thread via GitHub
yashmayya merged PR #14825: URL: https://github.com/apache/kafka/pull/14825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apach

Re: [PR] KAFKA-15888: DistributedHerder log context should not use the same client ID for each Connect worker by default [kafka]

2023-11-23 Thread via GitHub
yashmayya commented on PR #14825: URL: https://github.com/apache/kafka/pull/14825#issuecomment-1824105300 Test failures are unrelated, and all Connect tests pass locally. Merging to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Resolved] (KAFKA-15555) Ensure wakeups are handled correctly in PrototypeAsyncConsumer.poll()

2023-11-23 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-1. --- Resolution: Fixed > Ensure wakeups are handled correctly in PrototypeAsyncConsumer.poll() >

Re: [PR] KAFKA-15555: Ensure wakeups are handled correctly in poll() [kafka]

2023-11-23 Thread via GitHub
cadonna merged PR #14746: URL: https://github.com/apache/kafka/pull/14746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-15555: Ensure wakeups are handled correctly in poll() [kafka]

2023-11-23 Thread via GitHub
cadonna commented on PR #14746: URL: https://github.com/apache/kafka/pull/14746#issuecomment-1824090907 Build failures are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] MINOR: Add Uniform assignor to the default config [kafka]

2023-11-23 Thread via GitHub
dajac opened a new pull request, #14826: URL: https://github.com/apache/kafka/pull/14826 This patch adds the `Uniform` assignor to the default list of supported assignors. It also do small changes in the code. ### Committer Checklist (excluded from commit message) - [ ] Verify desi

Re: [PR] HOTFIX: Fix compilation error with Scala 2.12 [kafka]

2023-11-23 Thread via GitHub
mimaison merged PR #14823: URL: https://github.com/apache/kafka/pull/14823 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-11-23 Thread via GitHub
vamossagar12 commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1402217667 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -829,21 +902,50 @@ private CoordinatorResult consumerGr

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-11-23 Thread via GitHub
vamossagar12 commented on PR #14432: URL: https://github.com/apache/kafka/pull/14432#issuecomment-1824047001 Thanks @dajac . I have addressed all the comments. The second part of [this](https://github.com/apache/kafka/pull/14432/files#r1397382111) comment, I am not sure which new data struc

Re: [PR] KAFKA-15484: General Rack Aware Assignor [kafka]

2023-11-23 Thread via GitHub
dajac merged PR #14481: URL: https://github.com/apache/kafka/pull/14481 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Resolved] (KAFKA-15484) Implement general uniform broker side assignor

2023-11-23 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-15484. - Fix Version/s: 3.7.0 Resolution: Fixed > Implement general uniform broker side assignor >

[jira] [Created] (KAFKA-15889) Make General Assignor's rebalance incremental

2023-11-23 Thread David Jacot (Jira)
David Jacot created KAFKA-15889: --- Summary: Make General Assignor's rebalance incremental Key: KAFKA-15889 URL: https://issues.apache.org/jira/browse/KAFKA-15889 Project: Kafka Issue Type: Sub-t

Re: [PR] KAFKA-15484: General Rack Aware Assignor [kafka]

2023-11-23 Thread via GitHub
dajac commented on code in PR #14481: URL: https://github.com/apache/kafka/pull/14481#discussion_r1403102326 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/GeneralUniformAssignmentBuilder.java: ## @@ -14,17 +14,903 @@ * See the License for the s

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-23 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1403093845 ## core/src/test/scala/kafka/utils/TestInfoUtils.scala: ## @@ -39,6 +39,12 @@ object TestInfoUtils { } else { throw new RuntimeException(s"Unkn

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-23 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1403092072 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala: ## @@ -277,8 +294,10 @@ class PlaintextConsumerTest extends BaseConsumerTest { a

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-23 Thread via GitHub
AndrewJSchofield commented on PR #14758: URL: https://github.com/apache/kafka/pull/14758#issuecomment-1824026135 @dajac I suggest you simply merge #14871 and let me deal with the fall-out. This one is not going to be far behind. -- This is an automated message from the Apache Git Service.