[jira] [Created] (KAFKA-13952) Infinite retry timeout is not working

Jakub Malek created KAFKA-13952: --- Summary: Infinite retry timeout is not working Key: KAFKA-13952 URL: https://issues.apache.org/jira/browse/KAFKA-13952 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-827: Expose logdirs total and usable space via Kafka API

Hi, Thanks for the votes and feedback! The vote passes with: - 4 +1 (binding) votes from Luke, Tom, Jun and John - 3 +1 (non-binding) votes from Igor, Divij and Federico Thanks, Mickael On Wed, Jun 1, 2022 at 3:31 AM John Roesler wrote: > > Thanks for the KIP Mickael, > > I'm +1 (binding) > >

Re: [DISCUSS] KIP-846: Task-level Streams metrics for bytes/records Produced

> > Just a quick question: for filling the gap of sub-topology visibilities, > would task-level produced metrics be sufficient? If I understand your question correctly, you're asking whether we could just report at the task/subtopology level since we mainly want the bytes/throughput produced by t

Re: [DISCUSS] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

Hi David, If you are adding topic IDs, should we remove topic names? Ismael On Tue, May 31, 2022, 6:36 AM David Jacot wrote: > Hi all, > > Thanks for your feedback. > > I just updated the KIP as follows: > * I propose to add a `TopicId` field at the same time while we are bumping > the AlterPa

[VOTE] KIP-840: Config file option for MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer

Hi everyone! I propose to start voting for KIP-840: https://cwiki.apache.org/confluence/x/bBqhD Thanks, -- Alex

Re: [VOTE] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

Hi Colin, Thanks for your feedback! Please find my answers below. > However, I wonder if this will be feasible for ZK-based brokers to do. We > still support pre-topic-id IBP versions there, right? This might end up being > more complex than you were hoping. That's right. We support pre-topic-

Re: [DISCUSS] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

Hi Ismael, That's the plan. I just noticed that I forgot to change the version of the field in the AlterPartitionResponse. Let me fix that. Thanks, David On Wed, Jun 1, 2022 at 10:40 AM Ismael Juma wrote: > > Hi David, > > If you are adding topic IDs, should we remove topic names? > > Ismael >

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #965

See

Re: [VOTE] KIP-834: Pause / Resume KafkaStreams Topologies

Hi all, While reviewing my PR for KIP-834, Bruno noticed a case that we may not have discussed enough.* During the discussion, we decided that standby tasks would be paused. In order to do this, there are changes to the StoreChangelogReader around where it does restorations. Bruno noticed that

[GitHub] [kafka-site] ZachLC opened a new pull request, #413: Add Covage to powered-by

ZachLC opened a new pull request, #413: URL: https://github.com/apache/kafka-site/pull/413 We really are fans at Covage. congrats of what you've done with Kafka.  -- This

Re: [DISCUSS] KIP-846: Task-level Streams metrics for bytes/records Produced

I just want to send out a small update -- I decided to include the "- *consumed*" metrics in the KIP alongside the *"-produced"* metrics after all, for reasons I address in the paragraph I added at the end of the motivation section. Please let me know if you have any questions or concerns Cheers,

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

So, we have multiple options in terms of names, at this point I actually liked John's suggestion to use addMetricIfAbsent or something along those lines. Regarding the deprecation of sensor/metric method, I am not sure... Would like to know others' thoughts. Thanks! Sagar. On Wed, Jun 1, 2022 at

Re: [VOTE] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

Thanks, David. With these changes, I am +1 (binding) Great to see this KIP moving forward! Colin On Wed, Jun 1, 2022, at 02:04, David Jacot wrote: > Hi Colin, > > Thanks for your feedback! Please find my answers below. > >> However, I wonder if this will be feasible for ZK-based brokers to do.

Re: [DISCUSS] KIP-714: Client metrics and observability

Hi, Magnus, 51. Just to clarify my question. (1) Are standard metrics required for every client for this KIP to function? (2) Are we converting existing java metrics to the standard metrics and deprecating the old ones? If so, could we list all existing java metrics that need to be renamed and t

Re: [DISCUSS] KIP-827: Expose logdirs total and usable space via Kafka API

Thanks for the explanation. I think the question is that if we have disk utilization in our environment, what is the use case for KIP-827? The disk utilization in our environment can already do the job. Is there anything I missed? Thanks, Cong On Tue, May 31, 2022 at 2:57 AM Mickael Maison wrote

Re: [DISCUSS] KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft

Thanks for the updates to the KIP. I like enumerating invariants. Is it safe to say that if `InControlledShutdown` is true then `Fenced` must be false.

Re: [DISCUSS] KIP-846: Task-level Streams metrics for bytes/records Produced

Thanks Sophie, that makes sense. Also I agree that since we are adding it at finest granularity for consumed metrics, it's better to have symmetry and add produced metrics at processor-node level as well. Regarding the benchmarks, I think that would be critical to have before we add in the next re

Re: [VOTE] KIP-843: Adding metricOrElseCreate method to Metrics

I think `addMetricIfAbsent` is a good function name, and like John said people in the Java world are familiar with its return value semantics as well. Regarding deprecating the existing functions, I feel it is not necessary just for the function semantics difference between `sensors` and `metrics`

Re: [VOTE] KIP-834: Pause / Resume KafkaStreams Topologies

Hello Jim, I think If our primary goal would be to reduce resource utilization and potentially to stop the streaming pipeline for investigating possible bugs etc, then we should also pause active tasks' restoration as well since that 1) may still use resources, and 2) may load in bad data. Guozha

Re: [DISCUSS] KIP-844: Transactional State Stores

Hello Alex, > we flush the cache, but not the underlying state store. You're right. The ordering I mentioned above is actually: ... 3. producer.sendOffsetsToTransaction(); producer.commitTransaction(); 4. flush store, make sure all writes are persisted. 5. Update the checkpoint file to 200. But

Re: [DISCUSS] KIP-844: Transactional State Stores

Hi Guozhang, But I'm still trying to clarify how it guarantees EOS, and it seems that we > would achieve it by enforcing to not persist any data written within this > transaction until step 4. Is that correct? This is correct. Both alternatives - in-memory WriteBatchWithIndex and transactionalit

Re: [DISCUSS] KIP-844: Transactional State Stores

Alex, Thanks for your replies! That is very helpful. Just to broaden our discussions a bit here, I think there are some other approaches in parallel to the idea of "enforce to only persist upon explicit flush" and I'd like to throw one here -- not really advocating it, but just for us to compare

Re: [VOTE] KIP-746: Revise KRaft Metadata Records

Hi all, I updated this with the changes to FeatureLevelRecord that we did in KIP-778. Since the original version was never implemented, it would have been confusing to leave it there, I think. best, Colin On Wed, Jul 21, 2021, at 17:41, Colin McCabe wrote: > Hi all, > > I made an addendum to t

Re: [DISCUSS] KIP-846: Task-level Streams metrics for bytes/records Produced

Thanks Guozhang -- I'll definitely make sure we have this benchmarked with an eye for any regressions before it makes it into a release. That said, if it's any comfort, we use the cached system time and record only count and sum type metrics, so unlike the *-rate *metrics for example we don't have

Re: [VOTE] KIP-834: Pause / Resume KafkaStreams Topologies

Hey Jim, thanks for the update. I'm on the same side as Guozhang here, as I've expressed during the original discussion I think it would be confusing and possibly harmful to continue *any* kind of processing or action within Streams while it is "paused". In fact I sort of assumed we were including

[jira] [Created] (KAFKA-13953) kafka Console consumer fails with CorruptRecordException

Aldan Brito created KAFKA-13953: --- Summary: kafka Console consumer fails with CorruptRecordException Key: KAFKA-13953 URL: https://issues.apache.org/jira/browse/KAFKA-13953 Project: Kafka Issue

Re: [VOTE] KIP-846: Processor-level Streams metrics for records/bytes Producedd

+1 (binding) Thanks Sophie! On Tue, May 31, 2022 at 11:28 AM Walker Carlson wrote: > +1 non binding > > On Tue, May 31, 2022 at 12:19 PM John Roesler wrote: > > > +1 (binding) > > > > Thanks, > > John > > > > On Mon, May 30, 2022, at 13:00, Bill Bejeck wrote: > > > +1 (binding) > > > > > > -Bi