[jira] [Created] (KAFKA-17106) Enable KafkaConsumerTest#testFetchProgressWithMissingPartitionPosition

Chia-Ping Tsai created KAFKA-17106: -- Summary: Enable KafkaConsumerTest#testFetchProgressWithMissingPartitionPosition Key: KAFKA-17106 URL: https://issues.apache.org/jira/browse/KAFKA-17106 Project: K

[jira] [Resolved] (KAFKA-17016) Align the behavior of GaugeWrapper and MeterWrapper

[ https://issues.apache.org/jira/browse/KAFKA-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17016. Fix Version/s: 3.9.0 Resolution: Fixed > Align the behavior of GaugeWrapper and Met

[jira] [Resolved] (KAFKA-16672) Fix flaky DedicatedMirrorIntegrationTest.testMultiNodeCluster

[ https://issues.apache.org/jira/browse/KAFKA-16672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16672. Fix Version/s: 3.9.0 Resolution: Fixed > Fix flaky DedicatedMirrorIntegrationTest.t

[jira] [Created] (KAFKA-17107) Backport of PR-15327

Kartik Goyal created KAFKA-17107: Summary: Backport of PR-15327 Key: KAFKA-17107 URL: https://issues.apache.org/jira/browse/KAFKA-17107 Project: Kafka Issue Type: Bug Affects Versions: 3.

[jira] [Created] (KAFKA-17108) ListOffset API changes

Abhijeet Kumar created KAFKA-17108: -- Summary: ListOffset API changes Key: KAFKA-17108 URL: https://issues.apache.org/jira/browse/KAFKA-17108 Project: Kafka Issue Type: Sub-task R

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3091

See

Re: [DISCUSS] KIP-1066: Mechanism to cordon brokers and log directories

Hi Mickael, Thanks for the response. > 4. Cordoned log directories are persisted to the metadata log via the RegisterBrokerRecord, BrokerRegistrationChangeRecord records. If a broker is offline, the controller will use the latest known state of the broker to determine the broker's cordoned log di

Re: [DISCUSS] KIP-1032: Upgrade to Jakarta and JavaEE 9 in Kafka 4.0

The KIP has been updated to include the changes to the ConnectRestExtensionContext. On Tue, Jul 9, 2024 at 7:16 PM Christopher Shannon < christopher.l.shan...@gmail.com> wrote: > Thanks, I missed that, I can update the KIP tomorrow to include that. > > Chris > > On Tue, Jul 9, 2024 at 6:15 PM Chr

Re: [DISCUSS] KIP-1067: Remove ReplicaVerificationTool in 4.0 (deprecate in 3.9)

Hi all, @Tsai I have no strong opinion on the removal schedule. Since the policy is decided this way, I just updated the KIP from 4.0 to 5.0. (see the 'Final Notes'!) @Matthias @Justine If you are satisfied with this plan, please cast +1 to the voting thread: https://lists.apache.org/thread/b5

[jira] [Resolved] (KAFKA-17093) KafkaConsumer.seekToEnd should return LSO

[ https://issues.apache.org/jira/browse/KAFKA-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-17093. -- Resolution: Not A Problem > KafkaConsumer.seekToEnd should return LSO > -

Re: [DISCUSS] KIP-1066: Mechanism to cordon brokers and log directories

Hi Chia-Ping, Question 1) Yes that's a good summary. I'd also add that managing cordoned log directories is intended to be done by cluster administrators who also know about operations in-progress or planned such as scaling or adding/removing log directories. In practice you can't expect users to

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3092

See

Re: [DISCUSS] Apache Kafka 3.8.0 release

Hi Josep, OMG, what happened while I could not be involved with the Kafka community? Thanks for digging down the whole situation. @Mickael I greatly appreciate your effort in finalizing the feature. It seems like I only have to re-boot the KIP-780. Thanks, Dongjin On Tue, Jul 9, 2024 at 12:52 A

Re: [DISCUSS] KIP-1066: Mechanism to cordon brokers and log directories

Hi Luke, 4. You're right this scenario can happen. In this case I think the broker should enforce its new state and not create the replica as all its log directories are now cordoned. The replica will be offline and an administrator would need to reassign it to another broker. I expect most users

[jira] [Created] (KAFKA-17109) Reduce log message load for failed locking

Bruno Cadonna created KAFKA-17109: - Summary: Reduce log message load for failed locking Key: KAFKA-17109 URL: https://issues.apache.org/jira/browse/KAFKA-17109 Project: Kafka Issue Type: Impr

Re: [DISCUSS] Apache Kafka 3.8.0 release

Hi Dongjin, It's great to see you back! I hope what we did with KIP-390 matches your expectations. Looking forward to seeing the reboot of KIP-780. Thanks, Mickael On Wed, Jul 10, 2024 at 4:21 PM Dongjin Lee wrote: > > Hi Josep, > > OMG, what happened while I could not be involved with the Kafk

[jira] [Created] (KAFKA-17110) Enable valid test case in KafkaConsumerTest for AsyncKafkaConsumer

PoAn Yang created KAFKA-17110: - Summary: Enable valid test case in KafkaConsumerTest for AsyncKafkaConsumer Key: KAFKA-17110 URL: https://issues.apache.org/jira/browse/KAFKA-17110 Project: Kafka

RE: [VOTE] KIP-1054: Support external schemas in JSONConverter

Hi Chris, We did the testing using the JAR that supports external schema functionality. This included specifying the external schema file in the Kafka properties file, as well as embedding an inline schema directly within the connector properties file . Thanks, Priyanka From: Chris Egerton D

[jira] [Resolved] (KAFKA-16737) Clean up KafkaConsumerTest TODOs enabling tests for new consumer

[ https://issues.apache.org/jira/browse/KAFKA-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-16737. Resolution: Duplicate > Clean up KafkaConsumerTest TODOs enabling tests for new consumer >

[DISCUSS] KAFKA-17094: How should unregistered broker nodes be handled in KRaft quorum state?

Hi all, As reported in KAFKA-17094 [1], to scale down KRaft-based broker nodes, they must first be unregistered via the Kafka Admin API. If a node is removed before being unregistered, it can't be listed for unregistration because the describeQuorum won't show inactive observer nodes. This happens

Re: [VOTE] KIP-1054: Support external schemas in JSONConverter

Hi Priyanka, The issue is that connector configurations can also be submitted via the REST API as JSON objects, and if a user tries to embed a schema directly in that configuration as a string, it'll be fairly ugly. This does not apply if using a properties file (in standalone mode) or the config

[jira] [Created] (KAFKA-17111) ServiceConfigurationError in JsonSerializer/Deserializer during Plugin Discovery

Vikas Balani created KAFKA-17111: Summary: ServiceConfigurationError in JsonSerializer/Deserializer during Plugin Discovery Key: KAFKA-17111 URL: https://issues.apache.org/jira/browse/KAFKA-17111 Proj

Re: [VOTE] KIP-1067: Remove ReplicaVerificationTool in 4.0 (deprecate in 3.9)

+1 (binding) Thanks, Justine On Mon, Jul 8, 2024 at 1:59 AM Chia-Ping Tsai wrote: > > > > Note that we already have this tracker for tools deprecations, but I'm > > fine to have a dedicated one for this tool (maybe we can link them). > > https://issues.apache.org/jira/browse/KAFKA-14705. > > >

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hi all, This is the link to the KIP document. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1068%3A+New+JMX+metrics+for+the+new+KafkaConsumer Any comment is appreciated, On Tue, Jul 9, 2024 at 10:14 AM Brenden Deluna wrote: > Hello everyone, > > I would like to start the discussion th

[jira] [Created] (KAFKA-17112) StreamThread shutdown calls completeShutdown only in CREATED state

Ao Li created KAFKA-17112: - Summary: StreamThread shutdown calls completeShutdown only in CREATED state Key: KAFKA-17112 URL: https://issues.apache.org/jira/browse/KAFKA-17112 Project: Kafka Issue T

[jira] [Resolved] (KAFKA-17090) Add documentation to CreateTopicsResult#config to remind users that both "type" and "document" are null

[ https://issues.apache.org/jira/browse/KAFKA-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17090. Fix Version/s: 3.9.0 Resolution: Fixed > Add documentation to CreateTopicsResult#co

[jira] [Created] (KAFKA-17113) Flaky Test in GlobalStreamThreadTest#shouldThrowStreamsExceptionOnStartupIfExceptionOccurred

Ao Li created KAFKA-17113: - Summary: Flaky Test in GlobalStreamThreadTest#shouldThrowStreamsExceptionOnStartupIfExceptionOccurred Key: KAFKA-17113 URL: https://issues.apache.org/jira/browse/KAFKA-17113 Projec

[jira] [Created] (KAFKA-17114) handleRuntimeException

Ao Li created KAFKA-17114: - Summary: handleRuntimeException Key: KAFKA-17114 URL: https://issues.apache.org/jira/browse/KAFKA-17114 Project: Kafka Issue Type: Bug Reporter: Ao Li --

[jira] [Created] (KAFKA-17115) Closing newly-created consumers during rebalance can cause rebalances to hang

Chris Egerton created KAFKA-17115: - Summary: Closing newly-created consumers during rebalance can cause rebalances to hang Key: KAFKA-17115 URL: https://issues.apache.org/jira/browse/KAFKA-17115 Proje

Re: [DISCUSS] Apache Kafka 3.8.0 release

Hi Josep, A contributor just raised a regression [1] that I think should be addressed in 3.8.0 prior to the release. Summary: This change [2] causes multiple ERROR logs to appear during worker startup when operators install other, unrelated plugins that package the Jackson library. Severity: Work

[jira] [Created] (KAFKA-17116) New consumer may not send leave group if member ID received after close

Lianet Magrans created KAFKA-17116: -- Summary: New consumer may not send leave group if member ID received after close Key: KAFKA-17116 URL: https://issues.apache.org/jira/browse/KAFKA-17116 Project:

[jira] [Created] (KAFKA-17117) Avoid instantiating classpath plugins when service loading plugins

Greg Harris created KAFKA-17117: --- Summary: Avoid instantiating classpath plugins when service loading plugins Key: KAFKA-17117 URL: https://issues.apache.org/jira/browse/KAFKA-17117 Project: Kafka

Re: [DISCUSS] KIP-890 Server Side Defense

Hi, Justine, Thanks for the update and sorry for the late reply. 120. I am wondering what value is used for ClientTransactionProtocolVersion. Is it the version of the EndTxnRequest? 121. Earlier, you made the change to set lastProducerId in PREPARE to indicate that the marker is written for the

Re: [DISCUSS] KIP-890 Server Side Defense

Hey Jun, No worries. Work on this KIP has been blocked for a bit anyways -- catching up and rereading what I wrote :) 120. ClientTransactionProtocolVersion is the transaction version as defined by the highest transaction version (feature version value) supported by the client and the server. This

Re: [DISCUSS] KIP-890 Server Side Defense

Hi, Justine, Thanks for the reply. 120. If the broker sends TV Y for the finalized version in ApiVersionResponse, but the client doesn't support Y, how does the broker know the TV that the client supports? Jun On Wed, Jul 10, 2024 at 2:29 PM Justine Olshan wrote: > Hey Jun, > > No worries. Wo

Re: [DISCUSS] KIP-890 Server Side Defense

The client will send the newest EndTxn request version if and only if both the client and the server support kip-890 part 2. We set the value in the record based on the EndTxn version. Justine On Wed, Jul 10, 2024 at 2:50 PM Jun Rao wrote: > Hi, Justine, > > Thanks for the reply. > > 120. If th

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3093

See

Re: [DISCUSS] KIP-890 Server Side Defense

Sounds good, Justine. It would be useful to document that in the KIP. Thanks, Jun On Wed, Jul 10, 2024 at 2:59 PM Justine Olshan wrote: > The client will send the newest EndTxn request version if and only if both > the client and the server support kip-890 part 2. > We set the value in the rec

[jira] [Created] (KAFKA-17118) Remove StorageTool#buildMetadataProperties

Chia-Ping Tsai created KAFKA-17118: -- Summary: Remove StorageTool#buildMetadataProperties Key: KAFKA-17118 URL: https://issues.apache.org/jira/browse/KAFKA-17118 Project: Kafka Issue Type: Im

[jira] [Resolved] (KAFKA-17011) SupportedFeatures.MinVersion incorrectly blocks v0

[ https://issues.apache.org/jira/browse/KAFKA-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-17011. Resolution: Fixed > SupportedFeatures.MinVersion incorrectly blocks v0 > -

Re: [DISCUSS] KIP-655: Add deduplication processor in kafka-streams

Hi All, Thanks Ayoub for getting this KIP going again, I think a deduplication operator will be very useful. My applogies for being late to the discussion. Overall - I agree with the direction of the KIP but I have a couple of questions. 1. Regarding using offsets for tracking the first arr

[jira] [Resolved] (KAFKA-13183) Dropping nul key/value records upstream to repartiton topic not tracked via metrics

[ https://issues.apache.org/jira/browse/KAFKA-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13183. - Resolution: Fixed > Dropping nul key/value records upstream to repartiton topic not trac

[jira] [Reopened] (KAFKA-13183) Dropping nul key/value records upstream to repartiton topic not tracked via metrics

[ https://issues.apache.org/jira/browse/KAFKA-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-13183: - > Dropping nul key/value records upstream to repartiton topic not tracked via > metrics > -

[jira] [Created] (KAFKA-17119) After enabled kafka-ranger-plugin and baned the user for using describe in policy, but that user still can use describe.

StarBoy1005 created KAFKA-17119: --- Summary: After enabled kafka-ranger-plugin and baned the user for using describe in policy, but that user still can use describe. Key: KAFKA-17119 URL: https://issues.apache.org/jir

[jira] [Resolved] (KAFKA-13326) Add multi-cluster support to Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13326. - Resolution: Duplicate > Add multi-cluster support to Kafka Streams > ---

[jira] [Resolved] (KAFKA-8629) Kafka Streams Apps to support small native images through GraalVM

[ https://issues.apache.org/jira/browse/KAFKA-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8629. Resolution: Fixed GraalVM should work with KS now. > Kafka Streams Apps to support small n

Re: [DISCUSS] Apache Kafka 3.8.0 release

Hi Greg, I'll make sure the PR with the fix ( https://github.com/apache/kafka/pull/16565) gets merged today. Thanks for bringing this up! Best, -- Josep Prat Open Source Engineering Director, Aiven josep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexande