Request for contributor permission

Please could I have contributor permission to Apache Kafka. Here are my details: JIRA ID: schofielaj Wiki ID: schofielaj Thanks, Andrew Schofield IBM Event Streams

Re: transaction 2PC protocol

sees COMMIT for a transaction, it can deliver any messages it has held buffered for that transaction. It's not really sure whether COMMITTED was written successfully and fully replicated. It would generally be considered "good enough" to make this assumption. Andrew Schofield IBM E

Re: transaction 2PC protocol

make forward progress. If components restart, they carry on from where they left off. Andrew Schofield IBM Event Streams On 19/12/2018, 13:37, "Jose Raul Perez Rodriguez" wrote: Hi, thanks for the answer, it was helpful. So, if there are several topic-partitions in a t

Re: [VOTE] KIP-382 MirrorMaker 2.0

+1 (non-binding) Andrew Schofield IBM Event Streams On 21/12/2018, 01:23, "Srinivas Reddy" wrote: +1 (non binding) Thank you Ryan for the KIP, let me know if you need support in implementing it. - Srinivas - Typed on tiny keys. pls ignore typ

Re: [VOTE] KIP-389: Introduce a configurable consumer group size limit

+1 (non-binding) Looks like a good improvement. Andrew Schofield IBM Event Streams On 11/01/2019, 17:33, "Boyang Chen" wrote: +1 (non-binding) Thanks for the great work! Get Outlook for iOS<https://nam04.safelinks.protection.outlook.com/?url=https%3

[DISCUSS] KIP-419 Safely notify Kafka Connect SourceTask is stopped

Hi, I’ve created a new KIP to enhance the SourceTask interface in Kafka Connect. https://cwiki.apache.org/confluence/display/KAFKA/KIP-419:+Safely+notify+Kafka+Connect+SourceTask+is+stopped Comments welcome. Andrew Schofield IBM Event Streams

Re: [DISCUSS] KIP-419 Safely notify Kafka Connect SourceTask is stopped

rk is to make life easy for a connector developer and a nice clean "all done now" method would help. I think I'll add a diagram to illustrate to the KIP. Andrew Schofield IBM Event Streams On 18/01/2019, 19:02, "Ryanne Dolan" wrote: Andrew, do we know whether the SourceTa

Re: [VOTE] KIP-416: Notify SourceTask of ACK'd offsets, metadata

void commitRecord() { } } Or something like that. I do understand that the KIP mentions that recordLogged() is only called for records that are actually ACKed, but it's very similar in intent to commitRecord() in my view. Just my 2 cents. Andrew Schofield IBM Event Streams On 17/01/2

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

+1 (non-binding). Thanks for the KIP. On 21/01/2019, 12:45, "Eno Thereska" wrote: +1 (non binding). Thanks. On Mon, Jan 21, 2019 at 12:30 PM Mickael Maison wrote: > Bumping this thread. Considering this KIP is relatively straigh > forward, can we get some votes or

Re: [DISCUSS] KIP-419 Safely notify Kafka Connect SourceTask is stopped

ay to release this. Andrew Schofield IBM Event Streams On 21/01/2019, 10:13, "Andrew Schofield" wrote: Ryanne, Thanks for your comments. I think my overarching point is that the various states of a SourceTask and the transitions between them seem a bit loose and that makes it diff

Re: [DISCUSS] KIP-416: Notify SourceTask of ACK'd offsets, metadata

called when a record is ACKed and I would have thought that the connector implementor would want to provide only a single variant of commitRecord(). Andrew Schofield IBM Event Streams On 31/01/2019, 03:00, "Ryanne Dolan" wrote: I've updated the KIP and PR to overload commitRe

Re: [VOTE] KIP-412: Extend Admin API to support dynamic application log levels

Thanks for the KIP. +1 (non-binding) On 18/02/2019, 12:48, "Stanislav Kozlovski" wrote: Hey everybody, I'm starting a VOTE thread for KIP-412. This feature should significantly improve the flexibility and ease in debugging Kafka in run time KIP-412 - https://nam03.saf

Re: [VOTE] 2.2.0 RC2

+1 (non-binding) - Downloaded the artifacts - Ran Kafka Connect connectors Thanks, Andrew Schofield IBM Event Streams On 19/03/2019, 19:13, "Manikumar" wrote: +1 (non-binding) - Verified the artifacts, build from src, ran tests - Verified the quickstart, ra

[VOTE] KIP-419: Safely notify Kafka Connect SourceTask is stopped

: https://cwiki.apache.org/confluence/display/KAFKA/KIP-419%3A+Safely+notify+Kafka+Connect+SourceTask+is+stopped PR: https://github.com/apache/kafka/pull/6551 JIRA: https://issues.apache.org/jira/browse/KAFKA-7841 Thanks, Andrew Schofield IBM

Re: [VOTE] KIP-419: Safely notify Kafka Connect SourceTask is stopped

rceTask+is+stopped PR: https://github.com/apache/kafka/pull/6551 Thanks, Andrew Schofield IBM Event Streams On 15/04/2019, 15:59, "Edoardo Comar" wrote: Thanks Andrew. +1 (non-binding) -- Edoardo Comar

Re: [DISCUSS] KIP-419 Safely notify Kafka Connect SourceTask is stopped

> or if that may change in the future.) > > Ryanne > > On Thu, Jan 24, 2019 at 10:25 AM Ryanne Dolan > wrote: > > > > Andrew, I believe the task can be started again with start() during the > stopping and stopped states in yo

Re: [DISCUSS] KIP-419 Safely notify Kafka Connect SourceTask is stopped

ee that it would be nicer if the framework handled this for me, but the current API isn't too difficult to work around. Ryanne On Fri, May 3, 2019, 10:45 AM Andrew Schofield wrote: > Hi Vahid, > Thanks for taking a look at this KIP.

Re: [VOTE] KIP-419: Safely notify Kafka Connect SourceTask is stopped

It should be possible to get this small KIP into Kafka 2.3 but it's proving hard to get some binding votes. Please could the committers with background in Kafka Connect take a look and vote this week. Thanks, Andrew Schofield On 25/04/2019, 16:11, "Andrew Schofield" wrote:

Re: [VOTE] KIP-440: Extend Connect Converter to support headers

+1 (non-binding). Looks good. On 09/05/2019, 15:55, "Gwen Shapira" wrote: +1 (binding) Thank you! On Mon, May 6, 2019, 11:25 PM Yaroslav Tkachenko wrote: > Hi everyone, > > I'd like to start a vote for KIP-440: Extend Connect Converter to support > header

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

+1 (non-binding) Andrew Schofield On 05/06/2019, 14:04, "Ryanne Dolan" wrote: +1 (non-binding) Thanks Ryanne On Tue, Jun 4, 2019, 11:29 PM Cyrus Vafadari wrote: > Hi all, > > Like like to start voting in the following KIP: >

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

+1 (non-binding) Andrew On 06/06/2019, 15:15, "Ryanne Dolan" wrote: +1 (non-binding) Thanks Ryanne On Wed, Jun 5, 2019, 9:31 PM Satish Duggana wrote: > Thanks Viktor, proposed metrics are really useful to monitor replication > status on brokers.

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

> >>> who > > >>> > > disagree should definitely respond in the thread. > > >>> > > > > >>> > > Cyrus > > >>> > > > > >>> > > On Thu, Jun 6, 2019 at 11:16 AM Konstantine

Re: [VOTE] KIP-419: Safely notify Kafka Connect SourceTask is stopped

just knocking up a simple connector without worrying about thread management. Thanks, Andrew Schofield On 25/04/2019, 16:11, "Andrew Schofield" wrote: I'd like to encourage some more votes on KIP-419. It's a pretty small KIP to make it easier to handle resour

Re: [DISCUSS] KIP-317: Transparent Data Encryption

whether you'll manage to interest any committers in volunteering an opinion, and you'll need that before you can get the KIP accepted into Kafka. Thanks, Andrew Schofield (IBM) On 06/08/2019, 15:46, "Sönke Liebau" wrote: Hi, I have so far received prett

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

+1 (non-binding) On 09/08/2019, 08:39, "Sönke Liebau" wrote: +1 (non-binding) On Fri, 9 Aug 2019 at 04:45, Harsha Chintalapani wrote: > +1 (binding). much needed!! > > > On Thu, Aug 08, 2019 at 6:43 PM, Gwen Shapira wrote: > > > +1 (binding

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

Don't get me wrong. This is not an anti-Kafka rant. I just work with people used to traditional transactional systems, making use of Kafka for business applications, and it's important to understand the concepts on both sides and make sure your assumptions are valid. Andrew Schofield IBM Wats

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

-what guarantee for Kafka transactions? I think the key is whether the data which is asynchronously flushed is guaranteed to be recovered atomically in all cases. Asynchronous but atomic would be good. Andrew Schofield IBM Watson and Cloud Platform > > From: Guozhang Wang > Sent: 09 Decem

RE: [VOTE] KIP-51 - List Connectors REST API

+1 (non-binding) +1 for re-evaluating the KIP process. This one went through very quickly, but they can drag on sometimes. > Date: Wed, 23 Mar 2016 11:31:37 -0700 > Subject: Re: [VOTE] KIP-51 - List Connectors REST API > From: wangg...@gmail.com > To: dev

RE: [VOTE] KIP-43: Kafka SASL enhancements

understand the decision behind the >>>>> configuration parameters only allowing a single Login and >>>>> CallbackHandler class? This seems too limiting to me, and while >> Rajini >>>>> is trying hard to convince

RE: Interacting with a secured Kafka cluster via GSS-API

Wouldn't you use TLS to secure the connections? Encrypting just the credentials but not the connection seems brave. Andrew From: Dave Ariens To: "dev@kafka.apache.org" Date: 10/12/2015 15:43 Subject:RE: Interacting with a secured Kafka cluster via GSS-API > Is there a reas

RE: [DISCUSS] KIP-47 - Add timestamp-based log deletion policy

-based auto-expiration require use of message.timestamp.type=LogAppendTime? I think this KIP is a good idea, but I think it relies on strict ordering of timestamps to be workable. Andrew Schofield > Date: Fri, 12 Feb 2016 10:38:46 -0800 > Subject: Re: [DISCUSS] KIP-47 - Add timestamp

RE: [DISCUSS] Deprecating the old Scala producers for the next release

with deprecation annotations. It's just a marker that they're now living on borrowed time. * Remove the ability to connect from these deprecated clients two releases later - so I mean 0.12, not 0.10.0.2. Andrew Schofield > Subject: Re: [DISCUSS] Deprec

RE: [VOTE] KIP-43: Kafka SASL enhancements

+1 (non-binding) > From: ism...@juma.me.uk > Date: Mon, 7 Mar 2016 19:52:11 + > Subject: Re: [VOTE] KIP-43: Kafka SASL enhancements > To: dev@kafka.apache.org > > +1 (non-binding) > > On Thu, Mar 3, 2016 at 10:37 AM, Rajini Sivaram < > rajinisiva...@goo

RE: [DISCUSS] Time-based releases for Apache Kafka

ds. LTS works nicely for plenty of open-source projects. I think it would work well for Kafka too. Andrew Schofield > From: ofir.ma...@equalum.io > Date: Thu, 25 Aug 2016 16:07:07 +0300 > Subject: Re: [DISCUSS] Time-based releases for Apache K

RE: [DISCUSS] Time-based releases for Apache Kafka

pport releases proves too heavy, as you say we could reconsider. Andrew Schofield > From: g...@confluent.io > Date: Thu, 25 Aug 2016 09:57:30 -0700 > Subject: Re: [DISCUSS] Time-based releases for Apache Kafka > To: dev@kafka.apache.org > > I pre

Re: [DISCUSS] KIP-80: Kafka REST Server

ontributed to Apache Kafka so that it would be able to be developed in the same way. Andrew Schofield From: Suresh Srinivas Sent: 07 October 2016 22:41:52 To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-80: Kafka REST Server   ASF already gives us a clear framework and governance mode

KIP-382: MirrorMaker 2.0 progress to delivery?

that’s easiest with a broad range of people using it, and that’s only going to happen once it’s in a release. Does anyone have a view on the likelihood of it making 2.4? Thanks, Andrew Schofield

Re: [DISCUSS] KIP-416: Notify SourceTask of ACK'd offsets, metadata

I favour this approach too. Andrew Schofield On 01/10/2019, 09:15, "Ryanne Dolan" wrote: Thanks Randall, that works for me. Ryanne On Tue, Oct 1, 2019 at 9:09 AM Randall Hauch wrote: > Apologies for the late entry -- I entirely missed this KIP a

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

Thanks for the KIP. As you say, not that controversial. +1 (non-binding) Thanks, Andrew > On 25 Jul 2023, at 18:22, Hector Geraldino (BLOOMBERG/ 919 3RD A) > wrote: > > Hi everyone, > > The changes proposed by KIP-959 (Add BooleanConverter to Kafka Connect) have > a limited scope and shouldn'

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

Hi Jack, Thanks for the KIP. I have a few concerns about the idea. 1) I think that while a client-side partitioner seems like a neat idea and it’s an established part of Kafka, it’s one of the things which makes Kafka clients quite complicated. Just as KIP-848 is moving from client-side assignor

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

> we should ignore the keys in this case (similar to the effect of > setting *partitioner.ignore.keys > *config to false) and document it appropriately? > > Let me know what you think. > > Thanks! > Sagar. > > > On Thu, Jul 27, 2023 at 9:41 PM Andrew Schofield <

Re: [DISCUSS] KIP-714: Client metrics and observability

e good > to just log the INVALID_RECORD as an error, and treat the error as fatal > for the client (only fatal in terms of sending the metrics, the client can > keep functioning otherwise). What do you think? > > Thanks > Milind > > On Mon, Jul 24, 2023 at 8:18 PM Andrew Schofi

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

way to let client side >>> partitioning logic be present as well. Anyways, I am good on this point. >>> 2) The example provided does seem achievable by simply attaching the >>> partition number in the ProducerRecord. I guess if we can't find any >>> further example

[VOTE] KIP-714: Client metrics and observability

Hi, After almost 2 1/2 years in the making, I would like to call a vote for KIP-714 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability). This KIP aims to improve monitoring and troubleshooting of client performance by enabling clients to push metrics t

Re: [DISCUSS] KIP-714: Client metrics and observability

f aggregated data is too not helpful in the world of > telemetry 😊 > > 6. UnsupportedCompressionType: Shall we fallback to non-compression mode in > that case? I think compression is nice to have, but non-compressed > telemetry data is valuable as well. Especially for low throughpu

Re: [DISCUSS] Cluster-wide disablement of Tiered Storage

Hi Christo, I agree with you. Option 4.1 without a KIP seems like an acceptable starting point for something which will be relatively rare, provided that it’s easy for the user to get a list of the topics that have to be deleted before they can successfully start the broker with TS turned off.

Re: [VOTE] KIP-940: Broker extension point for validating record contents at produce time

Hi Adrian, Thanks for the KIP. Looks like a nice improvement. +1 (non-binding) Thanks, Andrew > On 2 Aug 2023, at 12:33, Adrian Preston wrote: > > Hello all, > > > > Edo and I would like to call for a vote on KIP-940: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-940%3A+Broker+exten

Re: [DISCUSS] KIP-714: Client metrics and observability

r way than the client side metrics > to identify this problem. > > Doguscan > > On Fri, 4 Aug 2023 at 15:33, Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > >> Hi Doguscan, >> Thanks for your comments. I’m glad to hear you’re interested in

Re: [DISCUSS] KIP-714: Client metrics and observability

remove or limit configured > metrics subscriptions. " How would a user know that the observed load was > due to handling metrics requests? > > 8. If I understand correctly, when the configured metrics reporter does not > implement the new interface the client would still follow the d

Re: [VOTE] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

Thanks for the KIP. Looks good to me. +1 (non-binding). Andrew > On 30 Aug 2023, at 18:07, Hector Geraldino (BLOOMBERG/ 919 3RD A) > wrote: > > This makes sense to me, +1 (non-binding) > > From: dev@kafka.apache.org At: 08/30/23 02:58:59 UTC-4:00To: > dev@kafka.apache.org > Subject: [VOTE] K

Re: [VOTE] KIP-714: Client metrics and observability

Bumping the voting thread for KIP-714. So far, we have: Non-binding +2 (Milind and Kirk), non-binding -1 (Ryanne) Thanks, Andrew > On 4 Aug 2023, at 09:45, Andrew Schofield wrote: > > Hi, > After almost 2 1/2 years in the making, I would like to call a vote for > KI

Re: [VOTE] KIP-714: Client metrics and observability

is already a busy channel, which leads to connection timeout and > subsequently disconnection. > > Thanks, > P > > On Fri, Sep 8, 2023 at 4:15 AM Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > >> Bumping the voting thread for KIP-714. >> >

Re: [DISCUSS] KIP-714: Client metrics and observability

sRequest periodically to preserve the current > behavior. > > 120. GetTelemetrySubscriptionsResponseV0 and PushTelemetryRequestV0: Could > we list error codes for each? > > 121. "ClientTelemetryReceiver.ClientTelemetryReceiver This method may be > called from the request

Re: [DISCUSS] KIP-714: Client metrics and observability

> correct? > > Thanks, > Kirk > >> On Sep 21, 2023, at 7:42 AM, Andrew Schofield >> wrote: >> >> Hi Jun, >> Thanks for your comments. I’ve updated the KIP to clarify where necessary. >> >> 110. Yes, agree. The motivation section mentions t

Re: [DISCUSS] KIP-714: Client metrics and observability

hink the client should actively > search for a broker that supports this RPC? As previously mentioned, the > broker uses the leastLoadedNode to find its first connection (am > I correct?), and what if that broker doesn't support the metric push? > > P > > On Fri,

Re: [DISCUSS] KIP-714: Client metrics and observability

n the consumer threading > refactor which is underway)". If I understand this correctly, it means > that KIP-714 won't work if the "old consumer" is used. Am I correct? > > Cheers, > David > > > On Fri, Sep 22, 2023 at 12:18 PM Andrew Schofield < >

Re: [DISCUSS] KIP-714: Client metrics and observability

? It set via > StreamsConfig, we would forward it to all clients automatically. > > > > > -Matthias > > > On 9/29/23 5:45 PM, David Jacot wrote: >> Hi Andrew, >> Thanks for driving this one. I haven't read all the KIP yet but I already >> have an initi

Re: [DISCUSS] KIP-714: Client metrics and observability

`Future` because collection > all the `instanceId` might be a blocking all on each client? I have already a > few idea how it could be implemented but I don't think it must be discussed > on the KIP, as it's an implementation detail. > > Thoughts? > > > -Matthia

Re: [DISCUSS] KIP-714: Client metrics and observability

p restoreConsumerInstanceIds(); >Map producerInstanceIds(); > } > > For the `threadKey`, we would use some pattern like this: > > [Stream|StateUpdater]Thread- > > > Would this work from your POV? > > > > -Matthias > > > On 10/9/23 2:15 AM, A

Re: [DISCUSS] KIP-932: Queues for Kafka

Hi Jack, Thanks for your comments. I have added a new section on Log Retention which describes the behaviour of the SPSO as the LSO advances. That makes total sense and was an omission from the KIP. I have added the other ideas as potential future work. I do like the idea of having the SPSO inf

Re: [DISCUSS] KIP-714: Client metrics and observability

very request type? >>>>> >>>>> Thanks, >>>>> >>>>> Jun >>>>> >>>>> On Wed, Oct 11, 2023 at 1:47 PM Matthias J. Sax >>>> wrote: >>>>> >>>>>> Thanks! >>>>>> >>>>>

Re: [DISCUSS] KIP-714: Client metrics and observability

ct 11, 2023 at 1:47 PM Matthias J. Sax wrote: > >> Thanks! >> >> On 10/10/23 11:31 PM, Andrew Schofield wrote: >>> Matthias, >>> Yes, I think that’s a sensible way forward and the interface you propose >> looks good. I’ll update the KIP accordingly.

Re: [DISCUSS] KIP-714: Client metrics and observability

aStreams initialization? >> 131. Why does globalConsumerInstanceId() return Optional while >> other consumer instances don't return Optional? >> 132. ClientMetricsSubscriptionRequestCount: Do we need this since we have a >> set of generic metrics >> (kafka.network:type=RequestMetrics

Re: [DISCUSS] KIP-714: Client metrics and observability

> listener. > > Jun > > On Thu, Oct 12, 2023 at 11:42 AM Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > >> Hi Jun, >> Thanks for your comments. >> >> 130. As Matthias described, and I am adding to the KIP, the >> `KafkaStreams#c

Re: KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp

Hi Roman, Thanks for the KIP. I think it’s an interesting idea, but I think the KIP document needs some more details added before it’s ready for review. For example, here’s a KIP in the same area which was delivered in an earlier version of Kafka. I think this is a good KIP to copy for a suitabl

Re: [VOTE] KIP-714: Client metrics and observability

ote: >>>>>>> >>>>>>> Hey Andrew - >>>>>>> >>>>>>> +1 but I don't have a binding vote! >>>>>>> >>>>>>> It took me a while to go through the KIP. Here are so

[DISCUSS] KIP-1000: List Client Metrics Configuration Resources

Hi, I would like to start discussion of a small KIP which fills a gap in the administration of client metrics configuration. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources Thanks, Andrew

Re: [DISCUSS] KIP-994: Minor Enhancements to ListTransactions and DescribeTransactions APIs

Hi Artem, I think you make a very good point. This also looks to me like it deserves a version bump for the request. Andrew > On 8 Nov 2023, at 04:12, Artem Livshits > wrote: > > Hi Raman, > > Thank you for the KIP. I think using the tagged field > in DescribeTransactionsResponse should be go

[VOTE] KIP-1000: List Client Metrics Configuration Resources

Hi, I’d like to start the voting for KIP-1000: List Client Metrics Configuration Resources. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources Thanks, Andrew

Re: [VOTE] KIP-1000: List Client Metrics Configuration Resources

istener in the new RPC. > As the client-metrics resource is not supported in Zookeeper mode hence > shouldn't we disallow ListClientMetricsResourcesRequest for > Zookeper in the APIVersion request itself? > > +1(non-binding) > > Regards, > Apoorv Mittal > +44 77216

Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources

Rao wrote: > > Hi, Andrew, > > Thanks for the KIP. Just one comment. > > Should we extend ConfigCommand or add a new tool to list client metrics? > > Thanks, > > Jun > > On Tue, Nov 7, 2023 at 9:42 AM Andrew Schofield < > andrew_schofield_j...@outlook.com> wro

Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources

for the KIP. Just one comment. >> >> Should we extend ConfigCommand or add a new tool to list client metrics? >> >> Thanks, >> >> Jun >> >> On Tue, Nov 7, 2023 at 9:42 AM Andrew Schofield < >> andrew_schofield_j...@outlook.com>

Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources

t-metrics.sh --bootstrap-server $BROKERS --describe > kafka-configs.sh --bootstrap-server $BROKERS --describe --entity-type > client-metrics > > Should we use --list instead to be more consistent with other tools like > (kafka-topics)? > > Thanks, > > Jun &g

Re: [VOTE] KIP-1000: List Client Metrics Configuration Resources

ew, >> >> Thanks for the KIP. +1 >> >> Jun >> >> On Thu, Nov 16, 2023 at 9:12 AM Andrew Schofield < >> andrew_schofield_j...@outlook.com> wrote: >> >>> Hi Apoorv, >>> Thanks for your vote. >>> >>> Initially, I put

Re: [DISCUSS] KIP-932: Queues for Kafka

mpacted topic, right? > > 143. So, the client sends DescribeShareGroupOffsets requests to GC, which > then forwards it to SC? > > 147. I guess a client only knows the rebalance triggered by itself, but not > the ones triggered by other members or topic/partition changes? &g

Re: [DISCUSS] KIP-932: Queues for Kafka

reUpdateValue. > > 158. ReadShareGroupOffsetsState: The state is the offsets. Should we rename > it to something like ReadShareGroupStartOffset? > > 159. members are assigned members round-robin => members are assigned > round-robin > > 160. "may called": ty

Re: [DISCUSS] KIP-1041: Drop `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8)

Hi David, I think this KIP is a very good idea. It would be good to get rid of this cruft. Thanks, Andrew > On 2 May 2024, at 18:54, David Jacot wrote: > > Hi folks, > > I have put together a very small KIP to > deprecate offsets.commit.required.acks in 3.8 and remove it in 4.0. See the > motiva

Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

Hi, I’ve changed my mind on this one having read through the comments. I don’t think the exception handler should be able to mess with the headers to the detriment of the code that called the handler. While I like the hygiene of having an ImmutableHeaders interface, I feel we can use the existing

Re: [DISCUSS] KIP-932: Queues for Kafka

ffsetsState and ReadShareGroupState. > The state in the latter is also offsets. > > Jun > > On Wed, May 1, 2024 at 9:51 PM Andrew Schofield > wrote: > >> Hi Jun, >> Thanks for your reply. >> >> 147. Perhaps the easiest is to take a look at the code in

Re: [VOTE] KIP-1036: Extend RecordDeserializationException exception

Hi Fred, Thanks for the KIP. It’s turned out nice and elegant I think. Definitely a worthwhile improvement. +1 (non-binding) Thanks, Andrew > On 30 Apr 2024, at 14:02, Frédérik Rouleau > wrote: > > Hi all, > > As there is no more activity for a while on the discuss thread, I think we > can st

Re: [DISCUSS] KIP-932: Queues for Kafka

ce time always 0? > > 150. I guess in the common case, there will be many more members than > partitions. So the need for (b) will be less common. We can probably leave > the persisting of the assignment out for now. > > 158. The new name sounds good to me. > > Jun > >

Re: [DISCUSS] KIP-932: Queues for Kafka

on KRaft clusters? For example, > the feature tool seems to be built only for the KRaft cluster. > > Jun > > On Fri, May 3, 2024 at 10:32 AM Andrew Schofield > wrote: > >> Hi Jun, >> Thanks for your reply. >> >> 147. Yes, I see what you mean. The

Re: [VOTE] KIP-1041: Drop `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8)

Hi, Thanks for the KIP. +1 (non-binding) Thanks, Andrew > On 8 May 2024, at 15:48, David Jacot wrote: > > Hi folks, > > I'd like to start a voting thread for KIP-1041: Drop > `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8). > > KIP: https://cwiki.apache.org/confluence/x/9YobEg >

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

Hi Alieh, Just a few more comments on the KIP. It is looking much less risky now the scope is tighter. [AJS1] It would be nice to have default implementations of the handle methods so an implementor would not need to implement both themselves. [AJS2] Producer configurations which are class names

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

gt;>> and then check the logs afterward for messages like "Error while >>> fetching >>>>> metadata with correlation id : >>>> {foo-topic=UNKNOWN_TOPIC_OR_PARTITION}". >>>>> >>>>> - I also don't think we n

Re: [VOTE] KIP-1038: Add Custom Error Handler to Producer

Hi Alieh, Thanks for the KIP. +1 (non-binding) Andrew > On 7 May 2024, at 16:56, Alieh Saeedi wrote: > > Hi all, > > It seems that we have no more comments, discussions, or feedback on > KIP-1038; therefore, I’d like to open voting for the KIP: Add Custom Error > Handler to Producer >

Re: [VOTE] KIP-932: Queues for Kafka

ar 2024 at 16:32, Kenneth Eversole >>>> wrote: >>>>> >>>>> Hi Andrew >>>>> >>>>> + 1 (Non-Binding) >>>>> >>>>> This will be great addition to Kafka >>>>> >>>>> On Mon, Ma

Re: [VOTE] KIP-932: Queues for Kafka

>> Best, >> David >> >> On Wed, May 15, 2024 at 10:37 PM Jun Rao wrote: >> >>> Hi, Andrew, >>> >>> Thanks for the update. Should we mark whether those metrics are >>> standard/required for KIP-714? >>> >>> Jun >>>

Re: [VOTE] KIP-932: Queues for Kafka

? > > Jun > > On Tue, May 14, 2024 at 7:31 AM Andrew Schofield > wrote: > >> Hi, >> I have made a small update to the KIP as a result of testing the new >> share consumer with client telemetry (KIP-714). >> >> I’ve added telemetry metric names t

Re: [VOTE] KIP-899: Allow producer and consumer clients to rebootstrap

>>>>> Hi, Ivan, >>>>> >>>>> Thanks for the KIP. +1 >>>>> >>>>> Just a minor comment. Should we add metadata.recovery.strategy to the >>>>> Streams and the newly introduced ShareConsumer (KIP-932) too? >>

Re: [VOTE] KIP 1047 - Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder

Thanks for the KIP. +1 (non-binding) Thanks, Andrew > On 23 May 2024, at 18:48, Chia-Ping Tsai wrote: > > > +1 > > Thanks for Yang to take over this! > >> Frank Yang 於 2024年5月24日 凌晨12:27 寫道： >> >> Hi all, >> >> I would like to start a vote on KIP-1047: Introduce new >> org.apache.kafka.tools.

[DISCUSS] KIP-1043: Administration of groups

Hi, I would like to start a discussion thread on KIP-1043: Administration of groups. This KIP enhances the command-line tools to make it easier to administer groups on clusters with a variety of types of groups. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1043%3A+Administration+of+grou

Re: [DISCUSS] KIP-1043: Administration of groups

d fails with a message indicating that > the group type is incorrect, rather than the existing message that the group > does not exist.” That sounds like a change that could trip up some brittle > scripts somewhere, but I don’t know if that’s a serious problem. > > Thanks! > >> O

Re: [DISCUSS] KIP-1050: Consistent error handling for Transactions

Hi Kaushik, Thanks for the KIP. This is definitely an area that needs clearing up so it’s good to see it. A few initial questions. 1. If I understand correctly, you are proposing to change the superclass of all subclasses of o.a.k.common.errors.ApiException which can be thrown by the producer o

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

Hi Chris, This KIP looks good to me. I particularly like the explanation of how the result will specifically check the worker health in ways that have previously caused trouble. Thanks, Andrew > On 7 Jun 2024, at 16:18, Mickael Maison wrote: > > Hi Chris, > > Happy Friday! The KIP looks good to

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

Hi Chris, Thanks for the KIP. +1 (non-binding) Thanks, Andrew > On 14 Jun 2024, at 15:48, Chris Egerton wrote: > > Hi all, > > Happy Friday! I'd like to kick off a vote on KIP-1017. > > Design doc: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+Health+check+endpoint+for+Kafka+C

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

Hi Alieh, Thanks for the KIP. I *really* don’t like adding a config which changes the behaviour of the flush() method. We already have too many configs. But I totally understand the problem that you’re trying to solve and some of the other suggestions in this thread seem neater. Personally, I wou

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

gt; Seems the option to use a config does not get a lot of support. >>>> >>>> So we need to go with some form or "overload / new method". I think >>>> Chris' point about not coupling it to `flush()` but rather >>>> `commitTransaction()` is actu

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

gt;>>>>> It's a fair question if this might be too subtle; to make it explicit, >>>>>> we could use `CommitOpions#ignorePendingSendErors()` [working name] to >>>>>> make it clear. >>>>>> >>>>>> >>>>

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

I > don't believe it's worth the increased cognitive load on users with > non-transactional producers to introduce an overloaded flush() variant. > > Cheers, > > Chris > > On Mon, Jun 24, 2024 at 9:39 AM Andrew Schofield > wrote: > >> Hi Alieh, >> Tha

  1   2   3   4   5   6   >