Re: [DISCUSS] KIP-816: Topology changes without local state reset

Hi Guozhang, Sorry I haven't had the time to respond to your earlier email, but I just wanted to clarify something with respect to your most recent email. My original plan in option A is to remove the entire Task ID from the State Store path, which would insulate it from any changes to the Task I

Re: [DISCUSS] KIP-822: Optimize the semantics of KafkaConsumer#pause to be consistent between the two RebalanceProtocols

Hi Guozhang, In short, what Riven wants to do, is that we keep the `pause` flag after rebalanced, while before this change, Rebalance does not preserve pause/resume state as described in KAFKA-2350 . In this case, do you think we can skip the KIP a

[jira] [Resolved] (KAFKA-13435) Static membership protocol should let the leader skip assignment (KIP-814)

[ https://issues.apache.org/jira/browse/KAFKA-13435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-13435. - Reviewer: Jason Gustafson Resolution: Fixed > Static membership protocol should let the le

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #688

See

Re: [DISCUSS] KIP-792: Add "generation" field into consumer protocol

Hi Guozhang, Thanks for your comments. I've updated the KIP. Here's what I've updated: * In the motivation section, I've added this paragraph after cooperativeStickyAssignor like this: *On the other hand, `StickyAssignor` is also adding "generation" field plus the "ownedPartitions" into subscri

[jira] [Resolved] (KAFKA-6823) Transient failure in DynamicBrokerReconfigurationTest.testThreadPoolResize

[ https://issues.apache.org/jira/browse/KAFKA-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-6823. -- Resolution: Fixed > Transient failure in DynamicBrokerReconfigurationTest.testThreadPoolResize

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #689

See

[GitHub] [kafka-site] bbejeck merged pull request #398: Update logo of Streaming Audio podcast

bbejeck merged pull request #398: URL: https://github.com/apache/kafka-site/pull/398 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr.

Re: [DISCUSS] KIP-816: Topology changes without local state reset

Hi Nick, When Guozgang and I were chatting, we realized that it’s not completely sufficient just to move the state store directories, because their names are not unique. In particular, more than one partition of the store may be assigned to the same instance. Right now, this is handled because

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #690

See

Re: [DISCUSS] KIP-792: Add "generation" field into consumer protocol

Thanks Luke, no more comments from me, nice work! On Mon, Feb 14, 2022 at 5:22 AM Luke Chen wrote: > Hi Guozhang, > > Thanks for your comments. I've updated the KIP. > Here's what I've updated: > > * In the motivation section, I've added this paragraph after > cooperativeStickyAssignor like this

Re: [DISCUSS] KIP-821: Connect Transforms support for nested structures

Hi Jorge, Thanks for the KIP! I'd love to see support for nested fields added to the out-of-the-box SMTs provided with Connect. Here are my initial thoughts: 1. I agree that there's a case to be made for expanding HoistField with a new config property for identifying a nested, to-be-hoisted field

Re: [VOTE] KIP-770: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

Actually @Guozhang, I just realised I didn't frame my question properly. What I wanted to know what, should this be a task-level metric or a cache-level metric. Place of definition is one thing, but what kind of metric should it be is what I wanted to know. Thanks! Sagar. On Mon, Feb 14, 2022 at

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

Got it. Thanks John, this make sense. I've updated the KIP to include the deprecation of: - KStream#transform - KStream#transformValues - KStream#flatTransform - KStream#flatTransformValues On Fri, 11 Feb 2022 at 15:16, John Roesler wrote: > Thanks, Jorge! > > I think it’ll be be

Re: [DISCUSS] KIP-820: Extend KStream process with new Processor API

No specific comments, but I just wanted to mention I like the direction of the KIP. My team is a big user of "transform" methods because of the ability to chain them, and I have always found the terminology challenging to explain alongside "process". It felt like one concept with two names. So mo

Re: [VOTE] KIP-770: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

Ah got it. I'd suggest we keep it a task-level metric then. On Mon, Feb 14, 2022 at 10:07 AM Sagar wrote: > Actually @Guozhang, I just realised I didn't frame my question properly. > What I wanted to know what, should this be a task-level metric or a > cache-level metric. Place of definition is

Re: [DISCUSS] KIP-816: Topology changes without local state reset

Thanks for the clarification John! Nick, sorry that I was not super clear in my latest email. I meant exactly what John said. Just to clarify, I do think that this KIP is relatively orthogonal to the named topology work; as long as we still keep the topo name encoded it should be fine since two n

Re: [DISCUSS] KIP-822: Optimize the semantics of KafkaConsumer#pause to be consistent between the two RebalanceProtocols

Hi Luke, Yeah my understanding of the KIP are from https://issues.apache.org/jira/browse/KAFKA-13463 and https://issues.apache.org/jira/browse/KAFKA-13425, and I think we and Riven are on the same page. If the changes are just within the internal classes, I think we can just treat it as a normal

Re: [DISCUSS] KIP-822: Optimize the semantics of KafkaConsumer#pause to be consistent between the two RebalanceProtocols

Apologies, but as worded KIP-822 / KAFKA-13425 are a bit challenging to follow. I've used pause() between version 0.10 and 2.7 with EAGER protocol only and have not yet observed cases where poll() returned messages unless resume() had been called. The KIP seems to contradict this though, and ind

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.0 #175

See Changes: -- [...truncated 348274 lines...] [2022-02-15T01:22:50.287Z] > Task :raft:testClasses UP-TO-DATE [2022-02-15T01:22:50.287Z] > Task :connect:json:testJar [2022-02-

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #691

See

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.1 #78

See