[jira] [Created] (KAFKA-14422) Consumer rebalance stuck after new static member joins a group with members not supporting static members

David Jacot created KAFKA-14422: --- Summary: Consumer rebalance stuck after new static member joins a group with members not supporting static members Key: KAFKA-14422 URL: https://issues.apache.org/jira/browse/KAFKA-

Re: [DISCUSS] KIP-878: Autoscaling for Statically Partitioned Streams

Hi Sophie, Thanks for the updates! I also feel the KIP is much cleaner now. I have one question: What is exactly the motivation behind metric num-autoscaling-failures? Actually, to realise that autoscaling did not work, we only need to monitor subtopology-parallelism over partition.autoscaling

[jira] [Created] (KAFKA-14423) Optimize CPU bottleneck caused by metrics capture

Divij Vaidya created KAFKA-14423: Summary: Optimize CPU bottleneck caused by metrics capture Key: KAFKA-14423 URL: https://issues.apache.org/jira/browse/KAFKA-14423 Project: Kafka Issue Type:

[jira] [Resolved] (KAFKA-14299) Benchmark and stabilize state updater

[ https://issues.apache.org/jira/browse/KAFKA-14299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy resolved KAFKA-14299. Assignee: Lucas Brutschy Resolution: Fixed > Benchmark and stabilize state updater >

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1378

See Changes: -- [...truncated 429918 lines...] [2022-11-28T14:51:07.337Z] [2022-11-28T14:51:07.337Z] Gradle Test Run :core:integrationTest > Gradle Test Executor 164 > Re

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi everyone, I've updated the KIP to resolve all the points that have been raised so far, with one exception: the ALOS default commit interval of 5 minutes is likely to cause WriteBatchWithIndex memory to grow too large. There's a couple of different things I can think of to solve this: - We

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

For now, I've settled on choosing an arbitrary default memory limit of 64MiB per-Task for buffering uncommitted records. I noticed that Kafka Streams already provides some arbitrary default configuration of RocksDB memory settings (i.e. memtable size etc.), and that many users will already be expli

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi all, Out of curiosity, why does the performance of the store degrade so significantly with the 844 implementation? I wouldn't be too surprised by a 50-60% drop (caused by each record being written twice), but 96% is extreme. The only thing I can think of which could create such a bottleneck wo

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi Colt, I didn't do any profiling, but the 844 implementation: - Writes uncommitted records to a temporary RocksDB instance - Since tombstones need to be flagged, all record values are prefixed with a value/tombstone marker. This necessitates a memory copy. - On-commit, iterate

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #1379

See

[jira] [Resolved] (KAFKA-14393) Change default size based retention for the cluster metadata partition

[ https://issues.apache.org/jira/browse/KAFKA-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Armando García Sancio resolved KAFKA-14393. Resolution: Fixed > Change default size based retention for the cl

[jira] [Resolved] (KAFKA-14303) Producer.send without record key and batch.size=0 goes into infinite loop

[ https://issues.apache.org/jira/browse/KAFKA-14303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-14303. - Fix Version/s: 3.4.0 3.3.2 Resolution: Fixed Merged the PR to 3.3 and trunk.

[jira] [Resolved] (KAFKA-14422) Consumer rebalance stuck after new static member joins a group with members not supporting static members

[ https://issues.apache.org/jira/browse/KAFKA-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14422. - Fix Version/s: 3.4.0 3.3.2 3.2.4 Resolution: Fixed

Re: [DISCUSS] KIP-889 Versioned State Stores

Thanks, Sagar and Bruno, for your insights and comments! > Sagar: Can we name according to the semantics that you want to support like `getAsOf` or something like that? I am not sure if we do that in our codebase though. Maybe the experts can chime in. Because it is a new method that will be adde

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1380

See

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.3 #126

See

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1381

See

Re: [DISCUSS] KIP-881: Rack-aware Partition Assignment for Kafka Consumers

I'm thinking of a case where the broker uses a plugin that does not use rack ids to determine client locality, in that case the consumer might arrive at the wrong decision based on rack ids. -Artem On Wed, Nov 23, 2022 at 3:54 AM Rajini Sivaram wrote: > Hi Artem, > > Thanks for reviewing the KI