[jira] [Created] (KAFKA-19187) MINOR: Disallow raw types and fail build for test files

Abhinav Dixit created KAFKA-19187: - Summary: MINOR: Disallow raw types and fail build for test files Key: KAFKA-19187 URL: https://issues.apache.org/jira/browse/KAFKA-19187 Project: Kafka Iss

Re: [SPAM]Re: [DISCUSS] KIP-1150 Diskless Topics

Hi! Thanks for the greate work and I'm excited to see it happens. These KIPs looks well to me. I have a question about the Batch Coordinator in KIP-1164. Seems Batch Coordinator is very important in the diskless implementation, could you explain more details on the implementation? For me, I'm w

[jira] [Created] (KAFKA-19186) Mark OffsetCommit and OffsetFetch APIs as stable

David Jacot created KAFKA-19186: --- Summary: Mark OffsetCommit and OffsetFetch APIs as stable Key: KAFKA-19186 URL: https://issues.apache.org/jira/browse/KAFKA-19186 Project: Kafka Issue Type: Su

Jenkins build is still unstable: Kafka » Kafka PowerPC Daily » test-powerpc #276

See

Re: [VOTE] KIP-1131: Improved controller-side monitoring of broker states

+1 (non-binding) Thanks, PoAn > On Apr 23, 2025, at 5:34 AM, Colin McCabe wrote: > > +1 (binding) > > thanks, > Colin > > On Tue, Apr 15, 2025, at 02:25, Federico Valeri wrote: >> +1 non binding >> >> Thanks >> >> On Tue, Mar 25, 2025 at 11:29 PM Kevin Wu wrote: >>> >>> Hello all, >>> >>

Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer

Hi Jiunn-Yang, Thanks for the updates! IMO, "allow.null.entries" might be a little too vague. Is "allow.null.offsets.entries" better? Thanks, Kirk On Tue, Apr 22, 2025, at 6:03 AM, 黃竣陽 wrote: > Hello everyone, > > I’ve submitted an updated version of this KIP based on recent discussions. > I

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #194

See

Re: [DISCUSS] KIP-1161: cleanup.policy shouldn't be empty

Hi, Jiunn-Yang, Regarding "The none policy will not delete or compact any segments", we should be more accurate. We won't delete segments based on log.retention.bytes/log.retention.ms, but we should continue to delete segments based on log.local.retention.bytes/log.retention.ms. Otherwise, we risk

Re: [VOTE] KIP-1131: Improved controller-side monitoring of broker states

+1 (binding) thanks, Colin On Tue, Apr 15, 2025, at 02:25, Federico Valeri wrote: > +1 non binding > > Thanks > > On Tue, Mar 25, 2025 at 11:29 PM Kevin Wu wrote: >> >> Hello all, >> >> I am manually bumping this thread. >> Any feedback or votes would be appreciated. >> >> Best regards, >> Kevin

Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

Hi Jun, The controller knows if there is metadata lost when it tries to write a new metadata image with the update MV. When the handleLoss callback is triggered, that indicates that metadata is lost. isBackwardsCompatible is not needed and will not be implemented for MV best, Colin On Tue, Ap

[DISCUSS] KIP-1166: Improve high-watermark replication

Hi all, I created a KIP to improve the latency for high-watermark replication. A few users have observed that their admin operations can take up 500ms before they are reflected in other parts of the system. This KIP is my attempt to resolve that issue: https://cwiki.apache.org/confluence/x/uIkEFQ

Re: [VOTE] KIP-1123: Rack-aware partitioning for Kafka Producer

+1 non-binding From: dev@kafka.apache.org At: 03/31/25 09:08:17 UTC-4:00To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-1123: Rack-aware partitioning for Kafka Producer +1 non binding On Mon, Mar 24, 2025 at 9:51 AM Frédérik Rouleau wrote: > > +1 (non-binding) > > On Sat, Mar 22, 2025 at 12:

Re: Custom Kafka Connect REST Extension for Token Authentication

Hi Jamie, We had success implanting basic authentication using out of the box constructs. Is this what you tried? https://docs.confluent.io/platform/current/security/authentication/http-basic-auth/overview.html We also implemented JWT but we did not have any success doing that out of the

Re: [DISCUSS] KIP-1139: Add support for OAuth jwt-bearer grant type

Also, Keycloak is MIT licensed. Is that OK to include in Kafka? On Tue, Apr 22, 2025, at 10:49 AM, Kirk True wrote: > Hi Manikumar, > > You mentioned using Keycloak for integration tests. Everything I'm seeing > online suggests that this is best done via Testcontainers. I don't see usage > of t

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #193

See

Re: [DISCUSS] KIP-1139: Add support for OAuth jwt-bearer grant type

Hi Jun, On Fri, Apr 18, 2025, at 10:19 AM, Jun Rao wrote: > Hi, Kirk, > > Thanks for the reply. > > JR1. If we have no clear usage, it's probably better to omit it. We could > always add a new param in the future if needed, right? OK, I'll remove the saslMechanism from the OAuthBearerConfigurab

Re: KIP-1155: Metadata Version Downgrades [DISCUSS]

Hi, Colin, Thanks for the KIP. During downgrade, how does the controller know whether there is metadata loss in the process? Do we still plan to implement the isBackwardsCompatible flag in MV as described in KIP-778? Jun On Mon, Apr 7, 2025 at 3:05 PM Colin McCabe wrote: > Hi all, > > I poste

Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics

Hi, Jiunn-Yang, Thanks for the improvement. KAFKA-18390 changes a public interface. Do we have a KIP for that? Jun On Tue, Apr 22, 2025 at 3:39 AM 黃竣陽 wrote: > Hello, > > I haven’t seen any feedback, so I’ll go ahead and implement the > improvement > regarding PluginMetrics#metricName. > > Be

Re: [DISCUSS] KIP-1139: Add support for OAuth jwt-bearer grant type

Hi Manikumar, You mentioned using Keycloak for integration tests. Everything I'm seeing online suggests that this is best done via Testcontainers. I don't see usage of that anywhere in the project thus far. Would adding a test dependency on Testcontainers be within the scope of this KIP, or sho

[jira] [Created] (KAFKA-19185) Logging Enhancements in the FETCH requests and their correlation IDs

Alieh Saeedi created KAFKA-19185: Summary: Logging Enhancements in the FETCH requests and their correlation IDs Key: KAFKA-19185 URL: https://issues.apache.org/jira/browse/KAFKA-19185 Project: Kafka

Re: [DISCUSS] KIP-1161: cleanup.policy shouldn't be empty

Hi, Jiunn-Yang, Thanks for the reply. Q2. What about existing empty values for group.coordinator.rebalance.protocols and process.roles during upgrade? Jun On Tue, Apr 22, 2025 at 7:29 AM 黃竣陽 wrote: > Hello Jun, > > Thanks for review this KIP. > > Q1 & Q3: > I’ve updated the method name accord

[DISCUSS] KIP-1162: Redesign ClientQuotaCallback#updateClusterMetadata

Hello everyone, I’d like to kick off a discussion about redesigning ClientQuotaCallback#updateClusterMetadata. As it stands, the current implementation is quite inefficient, since it requires creating a full Cluster object each time any change is made. On top of that, some of the information withi

[VOTE] KIP-1141: Simplifying MetadataQuorumCommand by Leveraging Admin API for Controller Management

Hello everyone, I would like to start a vote on KIP-1141, thank you! KIP: https://cwiki.apache.org/confluence/x/PZFEF Discussion thread: https://lists.apache.org/thread/mq5n8j8w0ycg9fhd1m4zcc2ctrgfd0d4 Best, Kuan-Po Tseng

Re: Custom Kafka Connect REST Extension for Token Authentication

Hi Jamie, This was an issue we ran into several years ago with the basic auth extension that comes OOTB with Connect. TL;DR: there are currently two endpoints that Connect uses for inter-worker, but intra-communication (i.e., REST requests that are made spontaneously from one worker to another, in

Re: [DISCUSS] KIP-1161: cleanup.policy shouldn't be empty

Hello Jun, Thanks for review this KIP. Q1 & Q3: I’ve updated the method name accordingly and revised the cleanup.policy documentation to clarify that the none policy cannot be used with any other policy. Q2: For users currently using an empty cleanup.policy, the approach is to apply the no

[jira] [Resolved] (KAFKA-19159) Remove time based eviction of share sessions from the cache

[ https://issues.apache.org/jira/browse/KAFKA-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apoorv Mittal resolved KAFKA-19159. --- Resolution: Fixed > Remove time based eviction of share sessions from the cache > --

Re: [DISCUSS] Change log.segment.bytes config type from int to long

Also here you can see guys have same idea https://forum.confluent.io/t/what-happens-if-i-increase-log-segment-bytes/5845 On Tue, 22 Apr 2025 at 15:41, Mikhail Fesenko wrote: > Currently we have big amount of topics and big data, and using high disk > storage, so rotate files too fast in case of

Re: [DISCUSS] Change log.segment.bytes config type from int to long

This is interesting, could be a huge barrier for this change, thanks for your clarification On Fri, 21 Feb 2025 at 18:52, Jun Rao wrote: > Hi, Mikhail, > > Currently, the index file uses 4 bytes to represent the file position of a > record batch within a segment file. This is based on the assump

Re: [DISCUSS] Change log.segment.bytes config type from int to long

Currently we have big amount of topics and big data, and using high disk storage, so rotate files too fast in case of high traffic means less sense, kafka has 50-100k open files index/offset/log so decrease amount of files/improve cache for them, it can give many improvements for big kafka topics

Re: [DISCUSS] KIP-1150 Diskless Topics

Hello! I want to start with saying that this is a big and impressive undertaking and I am really excited to see its progression! I am posting my initial comments in this thread, but they span a few of the child KIPs. Let me know which questions you would like to move elsewhere. I understand that y

Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer

Hello everyone, I’ve submitted an updated version of this KIP based on recent discussions. I'm happy to hear any further concerns or suggestions you might have. Best Regards, Jiunn-Yang > 黃竣陽 於 2025年4月22日 下午6:35 寫道： > > Hello, > > Thank for th

Re: [DISCUSS] KIP-1159: Large message reference based Serializer

Pump this thread up > On 11 Apr 2025, at 16:32, Omnia Ibrahim wrote: > > Thanks PoAn for having a look into the KIP >> PY_1: Does LargeMessageDeserializer need >> large.message.blob.store.blob.id_generator >> configuration? The blob id is encapsulated in Kafka event by >> LargeMessageSerialize

Re: [PR] Added missing `/migration` znode deletion on rollback KRaft migration [kafka-site]

mimaison commented on PR #684: URL: https://github.com/apache/kafka-site/pull/684#issuecomment-2820946520 Can you also open a PR against `apache/kafka:3.9` (https://github.com/apache/kafka/blob/3.9/docs/ops.html) with the same changes? Thanks -- This is an automated message from the Apac

Re: [PR] Added missing `/migration` znode deletion on rollback KRaft migration [kafka-site]

mimaison merged PR #684: URL: https://github.com/apache/kafka-site/pull/684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-19057) Stabilize KIP-932 Kafka protocol RPC definitions in AK 4.1

[ https://issues.apache.org/jira/browse/KAFKA-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-19057. -- Fix Version/s: 4.1.0 Resolution: Fixed > Stabilize KIP-932 Kafka protocol RPC d

Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics

Hello, I haven’t seen any feedback, so I’ll go ahead and implement the improvement regarding PluginMetrics#metricName. Best Regards, Jiunn-Yang > 黃竣陽 於 2025年4月14日 晚上11:43 寫道： > > Hi, > > There is an improvement in KIP-877. > > There will be an update to the PluginMetrics#metricName method

Re: [DISCUSS] KIP-1140: Avoid to return null value in Map from public api of consumer

Hello, Thank for the feedback. I will address this new/deprecated mechanism in the updated version of the KIP. Best Regards, Jiunn-Yang > Chia-Ping Tsai 於 2025年4月22日 中午12:58 寫道： > > hi Matthias > > Thanks for offering this approach. I had considered proposing it before. My > concern was the

Re: Custom Kafka Connect REST Extension for Token Authentication

Hi All,  Has anyone managed to get a REST extension working with multiple connect instances / workers? Many Thanks,  Jamie On Wednesday 16 April 2025 at 10:57:01 BST, Jamie wrote: Hi All,  I'm trying to implement a custom REST extension for Kafka Connect that allows a token to be use

Re: [VOTE] 3.9.1 RC0

Hi David, Thanks for the fix! I've tested it and it fixed the orphan commit issue. We'll close this VOTE thread and rebuild RC1 later. Thanks. Luke On Sun, Apr 20, 2025 at 12:28 PM TengYao Chi wrote: > Hi David > Thanks for the quick fix! > I will test this patch. > > Sincerely, > TengYao > >

[jira] [Resolved] (KAFKA-19166) Release tags are orphaned

[ https://issues.apache.org/jira/browse/KAFKA-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-19166. --- Fix Version/s: 3.9.1 Resolution: Fixed > Release tags are orphaned >