Re: What are the biggest issues with Apache Kafka?

2023-08-10 Thread Philip Nee
Hey Liam, I think apache jira would be a great place to find lower hanging fruits. We also have lots of flaky tests to resolve. What is you familiarity with Kafka? If you are new, i would suggest looking at the demos and examples in the repo, and maybe try to improve them. There is also a lot of

Re: What are the biggest issues with Apache Kafka?

2023-08-25 Thread Philip Nee
Hi Arpit: Here's a good starting point for Kraft-related issues: https://issues.apache.org/jira/browse/KAFKA-15401?jql=project%20%3D%20KAFKA%20AND%20component%20in%20(core%2C%20kraft) The recent Jenkin's builds have also been extremely flaky - This greatly impacts productivity. This would also be

Re: [VOTE] KIP-714: Client metrics and observability

2023-09-08 Thread Philip Nee
Hey Andrew - +1 but I don't have a binding vote! It took me a while to go through the KIP. Here are some of my notes during the reading: *Metrics* - Should we care about the client's leader epoch? There is a case where the user recreates the topic, but the consumer thinks it is still the same to

Re: [VOTE] KIP-714: Client metrics and observability

2023-09-13 Thread Philip Nee
cutive > pushes is best, but if you choose the “wrong” node to start with, it’s not > ideal. > > Thanks, > Andrew > > > On 8 Sep 2023, at 19:29, Philip Nee wrote: > > > > Hey Andrew - > > > > +1 but I don't have a binding vote! > > >

Re: [ANNOUNCE] New committer: Lucas Brutschy

2023-09-21 Thread Philip Nee
Nice! On Thu, Sep 21, 2023 at 9:21 AM Kirk True wrote: > Congratulations, Lucas! > > > On Sep 21, 2023, at 9:16 AM, Sagar wrote: > > > > Congrats Lucas ! > > > > On Thu, 21 Sep 2023 at 9:15 PM, Bruno Cadonna > wrote: > > > >> Hi all, > >> > >> The PMC of Apache Kafka is pleased to announce a n

Re: [DISCUSS] KIP-714: Client metrics and observability

2023-09-22 Thread Philip Nee
Hi Andrew - Question on top of your answers: Do you think the client should actively search for a broker that supports this RPC? As previously mentioned, the broker uses the leastLoadedNode to find its first connection (am I correct?), and what if that broker doesn't support the metric push? P O

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

2023-09-22 Thread Philip Nee
Congrats Justine! On Fri, Sep 22, 2023 at 7:07 PM Luke Chen wrote: > Hi, Everyone, > > Justine Olshan has been a Kafka committer since Dec. 2022. She has been > very active and instrumental to the community since becoming a committer. > It's my pleasure to announce that Justine is now a member o

Re: [DISCUSS] KIP-983: Full speed async processing during rebalance

2023-10-13 Thread Philip Nee
Hi Erik, Sorry for the delay, I have not finished reviewing the KIP, but I also have not forgotten about it! In general, KIP review process can be lengthy, so I think mailing list is the best bet to get the committer's attention. P On Fri, Oct 13, 2023 at 10:55 AM Erik van Oosten wrote: > Hi

Re: [DISCUSS] KIP-983: Full speed async processing during rebalance

2023-10-14 Thread Philip Nee
; > > Best, > > David > > > > Le ven. 13 oct. 2023 à 20:44, Erik van Oosten .invalid> > > a écrit : > > > >> Thanks Philip, > >> > >> No worries, I am not in a hurry. Knowing this is not forgotten is enough > >> for me. If there

Re: [DISCUSS] How to detect (and prevent) complex bugs in Kafka?

2023-10-24 Thread Philip Nee
Hi Divij, I have limited experience tackling bugs in Kafka, but for the issues I've fixed are usually found through..: 1. flaky unit/integration/system tests 2. Insufficient test coverage: When trying to add tests... 3. Related to #2 - Found through production at scale I have heard Kraft recently

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-13 Thread Philip Nee
Hi David et al, I agree with all the suggestions, but from what I've seen, the flaky tests tend to get ignored, and I'm afraid that disabling them would leave them getting forgotten. If the Jira ticket is accurate, we've got plenty of tickets opened for > 2 years

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2024-04-29 Thread Philip Nee
d also update the "examples" modules with > this new non-retriable exception. Wdyt? > > On Fri, Apr 21, 2023 at 11:13 PM Philip Nee wrote: > > > > Hey Jason, > > > > Thanks again. I've updated the KIP. > > > > P > > > > On Fri, Apr

Re: [VOTE] KIP-909: DNS Resolution Fallures Should Not Fail the Client

2024-04-29 Thread Philip Nee
he KIP! > > > > Regards, > > > > Rajini > > > > > > On Tue, Apr 25, 2023 at 8:52 PM Philip Nee wrote: > > > > > Thanks for the vote. We've decided to make a minor change to the > default > > > timeout from 5min to 2min. > > &

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

2024-07-10 Thread Philip Nee
Hi all, This is the link to the KIP document. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1068%3A+New+JMX+metrics+for+the+new+KafkaConsumer Any comment is appreciated, On Tue, Jul 9, 2024 at 10:14 AM Brenden Deluna wrote: > Hello everyone, > > I would like to start the discussion th

Re: [VOTE] KIP-1068: New metrics for the new KafkaConsumer

2024-07-25 Thread Philip Nee
Hi Brenden, Thanks for the KIP. +1 (non-binding). P On Thu, Jul 25, 2024 at 6:25 AM Lianet M. wrote: > Hi Brenden, > > +1 (non-binding) from me. > > Thanks for the KIP! > > Lianet > > On Thu, Jul 25, 2024 at 3:18 AM Bruno Cadonna wrote: > > > Hi Brenden, > > > > Thanks for the KIP! > > > > +1

Re: Kafka trunk test & build stability

2023-12-21 Thread Philip Nee
Hey Sophie - I've gotten 2 inflight PRs each with more than 15 retries... Namely: https://github.com/apache/kafka/pull/15023 and https://github.com/apache/kafka/pull/15035 justin filed a flaky test report here though: https://issues.apache.org/jira/browse/KAFKA-16045 P On Thu, Dec 21, 2023 at 3:

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2023-12-27 Thread Philip Nee
congrats divij! On Wed, Dec 27, 2023 at 8:55 AM Justine Olshan wrote: > Congratulations Divij! > > On Wed, Dec 27, 2023 at 4:20 AM Gaurav Narula wrote: > > > Congratulations Divij! > > > > Regards, > > Gaurav > > > > > On 27-Dec-2023, at 17:44, Mickael Maison > > wrote: > > > > > > Congratula

Re: [Question] About Kafka producer design decision making

2024-01-02 Thread Philip Nee
hey sean - a lot of uses of the Futures are in the public API and therefore take voting/effort to be changed. i don't know any reason for intentionally avoiding the use of CompletableFuture, however, others might know more than I do. thanks, P On Tue, Nov 14, 2023 at 1:27 AM 신수웅(Sean Sin) wrote

Re: Kafka consumer group crashing and not able to consume once service is up

2024-02-07 Thread Philip Nee
Hi Santhosh, Your problem statement confuses me a bit (apologize). You mentioned "if one of the kafka consumer(Service)" - Do you have a single member consumer group? Could you elaborate on the setup a bit? Did you also mean after restarting the "service", the service was not able to resume from c

Re: Kafka consumer group crashing and not able to consume once service is up

2024-02-08 Thread Philip Nee
Feb 7, 2024 at 11:49 PM Jain, Ankit wrote: > Update the below response. > > > > *From:* Marigowda, Santhosh Aditya > *Sent:* Thursday, February 8, 2024 1:14 PM > *To:* Philip Nee ; dev@kafka.apache.org > *Cc:* Jain, Ankit ; Namboodiri, Vishnu < > vishnu.nambood.

Re: [ANNOUNCE] New committer: Lianet Magrans

2024-08-28 Thread Philip Nee
congrats! On Wed, Aug 28, 2024 at 10:08 AM xijiu <422766...@qq.com.invalid> wrote: > Congratulations Lianet! > > > > > ---Original--- > From: "Kirk True" Date: Thu, Aug 29, 2024 01:01 AM > To: "dev" Subject: Re: [ANNOUNCE] New committer: Lianet Magrans > > > Congratulations Lianet > > On

Re: [ANNOUNCE] New committer: Jeff Kim

2024-09-10 Thread Philip Nee
congrats! On Tue, Sep 10, 2024 at 5:02 PM Kirk True wrote: > Congratulations Jeff! > > > On Sep 9, 2024, at 6:25 PM, Matthias J. Sax wrote: > > > > Congrats! > > > > On 9/9/24 12:34 PM, José Armando García Sancio wrote: > >> Congratulations Jeff! > >> On Mon, Sep 9, 2024 at 11:45 AM Justine Ols

KafkaConsumer refactor proposal

2022-09-13 Thread Philip Nee
Hi all, Here is the proposal to refactor the Kafka Consumer . The 1-pager is on the wiki, so please take a look at it. Also, this is a prerequisite for KIP-848 (the next gen rebalance protocol). Chee

Re: KafkaConsumer refactor proposal

2022-09-16 Thread Philip Nee
wrote: > Hi Philip! > > Thanks for the write-up. > > On Tue, Sep 13, 2022, at 2:13 PM, Philip Nee wrote: > > Hi all, > > > > Here is the proposal to refactor the Kafka Consumer > > < > https://cwiki.apache.org/confluence/display/KAFKA/Proposal%3A+Consumer

Re: KafkaConsumer refactor proposal

2022-09-16 Thread Philip Nee
` to also return > rebalance-listener-triggering events, but that may be less confusing. > > 2. I'd suggest for new modules like `ConsumerBackgroundThread`, we first > defines an interface in the `internals` package, e.g. `RequestEventHandler` > (assuming the previous rename sugge

Re: KafkaConsumer refactor proposal

2022-09-16 Thread Philip Nee
On Fri, Sep 16, 2022 at 3:01 PM Guozhang Wang wrote: > Hi Philip, > > On Fri, Sep 16, 2022 at 1:20 PM Philip Nee wrote: > > > Hi Guozhang, > > > > Thank you for the reviews, and I agree with your suggestions: If we find > > any behavior changes, we will

Re: KafkaConsumer refactor proposal

2022-09-18 Thread Philip Nee
w events > from the channel. > b. Run the state machine, and here are the following scenario: > c. Poll the networkClient. > d. Backoff for retryBackoffMs milliseconds. > > Thank you. > Again, really grateful for you to review the proposal, so thank you! P > L

Re: Regarding Producer Metric - "request-latency-avg"

2022-11-05 Thread Philip Nee
Hey Amrit, Could you elaborate on "this is not updated in producer metrics documentation in confluent"? The confluent documentation states, "The average request latency in ms", which seems right to me

Re: Regarding Producer Metric - "request-latency-avg"

2022-11-07 Thread Philip Nee
Hey Amrit, Actually, I think the confluent guide is correct because it seems like the request is created here . and the queue time

Re: Regarding an issue with message distributions in kafka.

2022-11-23 Thread Philip Nee
Hello Puneet, Why do you need multiple consumers from the same group consuming from the same partition? I am trying to think of a compiling reason for such a use case... It makes more sense to me to have a thread pool to process the records fetched from a single consumer. This is a great article

Re: [ANNOUNCE] New committer: Justine Olshan

2022-12-29 Thread Philip Nee
wow congrats! On Thu, Dec 29, 2022 at 1:05 PM Chris Egerton wrote: > Congrats, Justine! > > On Thu, Dec 29, 2022, 15:58 David Jacot wrote: > > > Hi all, > > > > The PMC of Apache Kafka is pleased to announce a new Kafka committer > > Justine > > Olshan. > > > > Justine has been contributing to

Reviews for KAFKA-13668

2023-01-17 Thread Philip Nee
Hey committers, Trying to get some attention to the KAFKA-13668. Is anyone familiar with the producer code able to provide feedback for the issue KAFKA-13668 ? In a nutshell, it allows producers to retry upon receiving authorization errors w/o needing t

Re: Pull request regarding to consumer metrics cleanup issue after shutdown

2023-01-17 Thread Philip Nee
Thanks Feiyan! Will spend some time reviewing the PR :) On Tue, Jan 17, 2023 at 7:08 PM Feiyan Yu wrote: > Howdy! > > I found that there is no logic to remove metrics registered in > `ConsumerCoordinatorMetrics` when `ConsumerCoordinator` is closed, which > might > > I created an issue related

Re: [DISCUSS] KIP-899: Allow clients to rebootstrap

2023-01-23 Thread Philip Nee
Hey Ivan, Thanks for the KIP. Some questions for clarification: It seems like the main problem is that if we don't poll frequently enough, the cluster topology can change entirely before the metadata is refreshed and thus causing staled clients. My question is: if you don't use the client for a lo

Re: Dead letter management

2023-01-31 Thread Philip Nee
Hey Viram, AFAIK, I don't think Kafka supports retries on dead messages. Your best bet would be to handle these cases by your application. Here's a useful article, in case you haven't read it: https://www.confluent.io/blog/spring-kafka-can-your-kafka-consumers-handle-a-poison-pill/#survive-a-pois

Requesting permission to the apache kafka project

2022-01-18 Thread Philip Nee
Hello, Would like to gain permission to the AK for contribution. wikiID: pnee jiraID: pnee Thanks, P

[DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-02-23 Thread Philip Nee
Hi all! I want to start a discussion thread about how we can handle client bootstrap failure due DNS lookup. This requires a bit of behavioral change, so a KIP is proposed and attached to this email. Let me know what you think! *A small remark here*: *As the title of this KIP might sound famili

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-02-23 Thread Philip Nee
d and move DNS resolution to poll, then KIP-899 becomes just a question > of whether we should call poll at other points after the first resolution. > It seems like these could potentially be merged into one proposal, or you > and Ivan could coordinate on symbiotic KIPs. > > Thanks agai

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-02-23 Thread Philip Nee
PM Ismael Juma wrote: > Thanks for the KIP. Not sure if I missed it, but how long will we retry for > and when do we give up and propagate the failure to the user? > > Ismael > > On Thu, Feb 23, 2023 at 9:30 AM Philip Nee wrote: > > > Hi all! > > > > I want

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-02-24 Thread Philip Nee
#x27;t really impact user-facing behavior. It also seems like > that detail might impact how intertwined this and KIP-899 are, though the > similarity could still be superficial either way. > > Cheers, > > Chris > > On Thu, Feb 23, 2023 at 9:21 PM Philip Nee wrote: >

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-02-27 Thread Philip Nee
1] - > > https://github.com/apache/kafka/blob/5f9d01668cae64b2cacd7872d82964fa78862aaf/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L606 > [2] - > > https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/run

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-02-28 Thread Philip Nee
nice to get input from the > >> community, though. > >> > >> I was toying with the idea of a NetworkException subclass too. It's a > >> simpler API, but it doesn't allow for preflight validation, which can be > >> useful in scenarios where submi

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-03-06 Thread Philip Nee
ill be helpful in any case. > > -Jason > > > > > > > On Tue, Feb 28, 2023 at 11:47 AM Philip Nee wrote: > > > Jason: > > Thanks for the feedback. Now giving it a second thought, I think your > > suggestion of logging the error might make sense, as I

Re: Should the mock consumer call the consumer rebalance listener on rebalance?

2023-03-23 Thread Philip Nee
Hey Dan, Thanks for looking into this. Are you talking about MockConsumer? If you invoke subscribe(Collection topics), it actually registers a Noop callback. Perhaps this is what you are seeing? P On Thu, Mar 23, 2023 at 11:11 AM Dan S wrote: > Hello all, > > It seems to me based on reading t

Re: Should the mock consumer call the consumer rebalance listener on rebalance?

2023-03-23 Thread Philip Nee
polling a few times, and then adding them > back. We're noticing our custom listener is never called, which was > unexpected, but based on the code analysis in the original email seems to > be the current implementation. The question is whether this is indeed > desired behavior. > &

Re: Should the mock consumer call the consumer rebalance listener on rebalance?

2023-03-23 Thread Philip Nee
Awesome! On Thu, Mar 23, 2023 at 12:31 PM Dan S wrote: > I'll pick it up, thanks! > > On Thu, Mar 23, 2023, 19:27 Philip Nee wrote: > > > Hey Dan, > > > > Your analysis looks right. I do see TODO item there to implement the > > rebalance callback. Would

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-04-05 Thread Philip Nee
a week or so. Link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-909%3A+DNS+Resolution+Failure+Should+Not+Fail+the+Clients Thanks, P On Thu, Mar 23, 2023 at 12:59 PM Philip Nee wrote: > Hey Kirk, > > Sorry about omitting your response; it slipped through the cracks... &

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-04-10 Thread Philip Nee
onnect/runtime/WorkerSinkTask.java#L606 > [2] - > > https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L439 > > Cheers, > > Chris > > On Fri, Feb 24, 2023 at 4:59 PM Philip Nee wrote: > > &

[VOTE] KIP-909: DNS Resolution Fallures Should Not Fail the Client

2023-04-10 Thread Philip Nee
Hey everyone! I'm starting a vote for KIP-909: DNS Resolution Fallures Should Not Fail the Client Please refer to the discussion thread here: https://lists.apache.org/thread/st84zzwnq5m3pkzd1r7jk9lmqdt9m98s Thanks! P

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-04-19 Thread Philip Nee
nue trying to reach the > bootstrap servers perhaps? That would be closer to behavior today which > only treats DNS resolution failures as fatal. What do you think? > > Best, > Jason > > On Mon, Apr 10, 2023 at 1:53 PM Philip Nee wrote: > > > Thanks, everyone: I'm st

Re: KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum

2023-04-19 Thread Philip Nee
Hey Colin, I still need to finish reading and understanding the KIP, but I have a couple of comments despite being ignorant of most of the KRaft stuff. (Sorry!) Firstly, does it make sense to create an extension of the current AdminClient only to handle these specific KRaft use cases? It seems cu

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-04-20 Thread Philip Nee
but connections fail. I know I'm the > one that suggested generalizing the configuration, but I feel some > hesitation after thinking about it more. > > Thanks, > Jason > > On Wed, Apr 19, 2023 at 8:10 PM Philip Nee wrote: > > > Hey Jason, > > > > Thanks

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

2023-04-21 Thread Philip Nee
Hey Jason, Thanks again. I've updated the KIP. P On Fri, Apr 21, 2023 at 9:56 AM Jason Gustafson wrote: > Hey Philip, that sounds good to me. > > -Jason > > > > On Thu, Apr 20, 2023 at 1:20 PM Philip Nee wrote: > > > Hey Jason, > > > > Thanks

Re: [VOTE] KIP-909: DNS Resolution Fallures Should Not Fail the Client

2023-04-25 Thread Philip Nee
t; > +1. Thanks for the design. Looking forward to the implementation. > > > > On Tue, Apr 25, 2023 at 10:49 AM Jason Gustafson > > wrote: > > > > > > +1 Thanks Philip! > > > > > > > > > On Thu, Apr 13, 2023 at 7:49 AM Kirk True wr

Re: [DISCUSS] Adding non-committers as Github collaborators

2023-04-28 Thread Philip Nee
Hi all, opinion from a regular person here: I think being able to re-trigger jenkin's test and tagging can be very helpful for a lot of times. On Fri, Apr 28, 2023 at 9:31 AM John Roesler wrote: > Hi all, > > This is a great suggestion! It seems like a really good way to make the > Apache Kafka

Re: [DISCUSS] Recommendations for managing long-running projects

2023-04-28 Thread Philip Nee
Hi Kirk and David, Thanks for raising this, Kirk. David, to your questions, as I'm also contributing to the project: Do you expect the changes to come in incrementally, or as a few large patches? We try to limit the PR size as much as possible, but sometimes it is inevitably large... We could be

Re: Kafka client needs KAFKA-10337 to cover async commit use case

2023-05-02 Thread Philip Nee
Hey Erik, Just a couple of questions to you: Firstly, could you explain the situation in that you would prefer to invoke commitAsync over commitSync in the rebalance listener? Typically we would use the synchronized method to ensure the commits are completed before moving on with the rebalancing,

Re: Kafka client needs KAFKA-10337 to cover async commit use case

2023-05-02 Thread Philip Nee
Sorry - I dug a bit into the old PR. Seems like the issue is there's broken contract that if the commitSync won't wait for the previous async commits to complete if it tries to commit an empty offset map. On Tue, May 2, 2023 at 12:49 PM Philip Nee wrote: > Hey Erik, > &g

Re: Kafka client needs KAFKA-10337 to cover async commit use case

2023-05-05 Thread Philip Nee
7;t wait for the previous async > >> commits > >> to complete if it tries to commit an empty offset map. > > > > Indeed! I am assuming the same is true for commitAsync. The important > > thing is that we need something to get those callbacks. I would prefer > &g

Re: Kafka client needs KAFKA-10337 to cover async commit use case

2023-05-05 Thread Philip Nee
gt; workaround. > > I guess we can fix this by adding `client.pollNoWakeup()`, e.g. at the > start of `ConsumerCoordinator.commitOffsetsAsync`. Is that an acceptable > change? > > Kind regards, > Erik. > > > Op 05-05-2023 om 20:20 schreef Philip Nee: > > He

Re: KafkaConsumer refactor proposal

2023-05-09 Thread Philip Nee
> background thread. In such cases, user should have the visibility to > identify that a need to scale the background thread processing is required. > > -- > Divij Vaidya > > > > On Sun, Sep 18, 2022 at 11:24 PM Philip Nee wrote: > > > On Sun, Sep 18, 2022 at 6:03 AM

Re: KafkaConsumer refactor proposal

2023-05-23 Thread Philip Nee
Hi all! I've added the "KIP-848" and "ctr" tags to the relevant PRs. see: https://github.com/apache/kafka/pulls?q=label%3Actr+sort%3Acreated-asc On Tue, May 9, 2023 at 10:29 AM Philip Nee wrote: > Hey Divij, > > Firstly, THANK YOU for reaching out and providi

Re: [DISCUSS] Regarding Old PRs

2023-06-01 Thread Philip Nee
Hey Josep, 2-3weeks is pretty optimistic, but anything over a year can probably be closed. If there's no response. If they don't or we can't get any answer, we could close the > PRs after 2 to 3 weeks. Regular contributors can't label their own PRs though - it requires a committer/collaborator t

Re: [ANNOUNCE] New committer: Divij Vaidya

2023-06-13 Thread Philip Nee
Congrats! On Tue, Jun 13, 2023 at 8:17 PM Randall Hauch wrote: > Congratulations! > > On Tue, Jun 13, 2023 at 12:48 PM Matthias J. Sax wrote: > > > Congrats! > > > > On 6/13/23 10:24 AM, Satish Duggana wrote: > > > Congratulations Divij!! > > > > > > On Tue, 13 Jun 2023 at 22:41, Manyanda Chiti

Re: KafkaConsumer refactor proposal

2023-07-10 Thread Philip Nee
er. > > Once we have reached a conclusion on the above points, perhaps, we can > create JIRAs to address them and work with the community members to resolve > them. > > [1] > > https://github.com/netty/netty/blob/57d536c59e434e88c0d8374d6b03671edb02885f/common/src/main/java

Re: [VOTE] KIP-944 Support async runtimes in consumer, votes needed!

2023-07-11 Thread Philip Nee
Hey Erik, Sorry for holding up this email for a few days since Colin's response includes some of my concerns. I'm in favor of this KIP, and I think your approach seems safe. Of course, I probably missed something therefore I think this KIP needs to cover different use cases to demonstrate it doe

Re: [VOTE] KIP-944 Support async runtimes in consumer, votes needed!

2023-07-11 Thread Philip Nee
converting this in to KIP-945. Thanks, P On Tue, Jul 11, 2023 at 10:33 AM Philip Nee wrote: > Hey Erik, > > Sorry for holding up this email for a few days since Colin's response > includes some of my concerns. I'm in favor of this KIP, and I think your > approach seems saf

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-14 Thread Philip Nee
Hey Mayank, Thanks for the KIP. I think this is a great proposal, and I'm in favor of this idea. A few comments: 1. Claiming metadata refresh is done asynchronously is misleading. The metadata refresh requires Network Client to be physically polled, which is done in a separate thread in Produce

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-17 Thread Philip Nee
ean the response for the Metadata RPC? I think brokers always > have a view of the cluster, although it can be stale,it would always return > a leader(whether old or new). > > Mayank > > On Fri, Jul 14, 2023 at 8:53 PM Philip Nee wrote: > > > Hey Mayank, > > >

Re: [ANNOUNCE] New committer: Andrew Schofield

2024-12-02 Thread Philip Nee
congrats! On Mon, Dec 2, 2024 at 9:28 AM Christo Lolov wrote: > Congratulations Andrew, well deserved! > > On Mon, 2 Dec 2024 at 16:36, Josep Prat > wrote: > > > Congrats Andrew! > > > > > > On Mon, Dec 2, 2024 at 5:33 PM Mickael Maison > > wrote: > > > > > Congratulations Andrew! > > > > > >

[jira] [Created] (KAFKA-15250) DefaultBackgroundThread is running tight loop

2023-07-25 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15250: -- Summary: DefaultBackgroundThread is running tight loop Key: KAFKA-15250 URL: https://issues.apache.org/jira/browse/KAFKA-15250 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-15304) CompletableApplicationEvents aren't being completed when the consumer is closing

2023-08-04 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15304: -- Summary: CompletableApplicationEvents aren't being completed when the consumer is closing Key: KAFKA-15304 URL: https://issues.apache.org/jira/browse/KAFKA-15304 Pr

[jira] [Created] (KAFKA-15305) The background thread should try to process the remaining task until the shutdown timer is expired

2023-08-04 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15305: -- Summary: The background thread should try to process the remaining task until the shutdown timer is expired Key: KAFKA-15305 URL: https://issues.apache.org/jira/browse/KAFKA-15305

[jira] [Created] (KAFKA-15333) Flaky build failure throwing Connect Exception: Could not connect to server....

2023-08-10 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15333: -- Summary: Flaky build failure throwing Connect Exception: Could not connect to server Key: KAFKA-15333 URL: https://issues.apache.org/jira/browse/KAFKA-15333 Project

[jira] [Created] (KAFKA-15474) AbstractCoordinator.testWakeupAfterSyncGroupReceivedExternalCompletion seems flaky

2023-09-18 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15474: -- Summary: AbstractCoordinator.testWakeupAfterSyncGroupReceivedExternalCompletion seems flaky Key: KAFKA-15474 URL: https://issues.apache.org/jira/browse/KAFKA-15474

[jira] [Created] (KAFKA-15475) Timeout request might retry forever even if the user API times out in AsyncConsumer

2023-09-18 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15475: -- Summary: Timeout request might retry forever even if the user API times out in AsyncConsumer Key: KAFKA-15475 URL: https://issues.apache.org/jira/browse/KAFKA-15475

[jira] [Created] (KAFKA-15531) Ensure coordinator node is removed upon disconnection exception

2023-10-03 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15531: -- Summary: Ensure coordinator node is removed upon disconnection exception Key: KAFKA-15531 URL: https://issues.apache.org/jira/browse/KAFKA-15531 Project: Kafka

[jira] [Created] (KAFKA-15533) Ensure HeartbeatRequestManager only send out some fields once

2023-10-03 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15533: -- Summary: Ensure HeartbeatRequestManager only send out some fields once Key: KAFKA-15533 URL: https://issues.apache.org/jira/browse/KAFKA-15533 Project: Kafka

[jira] [Created] (KAFKA-15534) Propagate client response time when timeout to the request handler

2023-10-03 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15534: -- Summary: Propagate client response time when timeout to the request handler Key: KAFKA-15534 URL: https://issues.apache.org/jira/browse/KAFKA-15534 Project: Kafka

[jira] [Created] (KAFKA-15548) Handling close() properly

2023-10-04 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15548: -- Summary: Handling close() properly Key: KAFKA-15548 URL: https://issues.apache.org/jira/browse/KAFKA-15548 Project: Kafka Issue Type: Bug Components

[jira] [Created] (KAFKA-15551) Evaluate conditions for short circuiting consumer API calls

2023-10-05 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15551: -- Summary: Evaluate conditions for short circuiting consumer API calls Key: KAFKA-15551 URL: https://issues.apache.org/jira/browse/KAFKA-15551 Project: Kafka

[jira] [Created] (KAFKA-15553) Review committed offset refresh logic

2023-10-05 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15553: -- Summary: Review committed offset refresh logic Key: KAFKA-15553 URL: https://issues.apache.org/jira/browse/KAFKA-15553 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-15562) CommitRequestManager needs to test different error handling

2023-10-06 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15562: -- Summary: CommitRequestManager needs to test different error handling Key: KAFKA-15562 URL: https://issues.apache.org/jira/browse/KAFKA-15562 Project: Kafka

[jira] [Resolved] (KAFKA-15278) Implement client support for KIP-848 ConsumerGroupHeartbeat protocol RPC

2023-10-09 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15278. Resolution: Fixed > Implement client support for KIP-848 ConsumerGroupHeartbeat protocol

[jira] [Created] (KAFKA-15588) Purge the unsent offset commits/fetches when the member is fenced

2023-10-11 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15588: -- Summary: Purge the unsent offset commits/fetches when the member is fenced Key: KAFKA-15588 URL: https://issues.apache.org/jira/browse/KAFKA-15588 Project: Kafka

[jira] [Created] (KAFKA-15592) Member does not need to always try to join a group when a groupId is configured

2023-10-11 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15592: -- Summary: Member does not need to always try to join a group when a groupId is configured Key: KAFKA-15592 URL: https://issues.apache.org/jira/browse/KAFKA-15592 Project

[jira] [Created] (KAFKA-15642) Ensure offset fetcher behaves correctly when the request is Timeout

2023-10-18 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15642: -- Summary: Ensure offset fetcher behaves correctly when the request is Timeout Key: KAFKA-15642 URL: https://issues.apache.org/jira/browse/KAFKA-15642 Project: Kafka

[jira] [Created] (KAFKA-15697) Add local assignor and ensure it cannot be used with server side assignor

2023-10-26 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15697: -- Summary: Add local assignor and ensure it cannot be used with server side assignor Key: KAFKA-15697 URL: https://issues.apache.org/jira/browse/KAFKA-15697 Project: Kafka

[jira] [Resolved] (KAFKA-15550) OffsetsForTimes validation for negative timestamps in new consumer

2023-10-27 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15550. Resolution: Fixed > OffsetsForTimes validation for negative timestamps in new consu

[jira] [Resolved] (KAFKA-15631) Do not send new heartbeat request while another one in-flight

2023-10-30 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15631. Resolution: Not A Problem > Do not send new heartbeat request while another one in-fli

[jira] [Created] (KAFKA-15773) Group protocol configuration should be validated

2023-11-02 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15773: -- Summary: Group protocol configuration should be validated Key: KAFKA-15773 URL: https://issues.apache.org/jira/browse/KAFKA-15773 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-15562) Ensure fetch offset and commit offset handler handles both timeout and various error types

2023-11-02 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15562. Resolution: Fixed > Ensure fetch offset and commit offset handler handles both timeout

[jira] [Resolved] (KAFKA-15316) CommitRequestManager not calling RequestState callbacks

2023-11-02 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15316. Resolution: Fixed > CommitRequestManager not calling RequestState callba

[jira] [Resolved] (KAFKA-15531) Ensure coordinator node is removed upon disconnection exception

2023-11-02 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15531. Resolution: Fixed > Ensure coordinator node is removed upon disconnection except

[jira] [Resolved] (KAFKA-15679) Client support for new consumer configs

2023-11-02 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15679. Resolution: Fixed > Client support for new consumer conf

[jira] [Created] (KAFKA-15775) Implement listTopics() and partitionFor() for the AsyncKafkaConsumer

2023-11-02 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15775: -- Summary: Implement listTopics() and partitionFor() for the AsyncKafkaConsumer Key: KAFKA-15775 URL: https://issues.apache.org/jira/browse/KAFKA-15775 Project: Kafka

[jira] [Resolved] (KAFKA-15686) Consumer should be able to detect network problem

2023-11-06 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-15686. Assignee: Philip Nee Resolution: Won't Fix The user can use the admin client to ge

[jira] [Created] (KAFKA-15803) Update last seen epcoh during commit

2023-11-09 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15803: -- Summary: Update last seen epcoh during commit Key: KAFKA-15803 URL: https://issues.apache.org/jira/browse/KAFKA-15803 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-15818) Implement max poll internval

2023-11-13 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15818: -- Summary: Implement max poll internval Key: KAFKA-15818 URL: https://issues.apache.org/jira/browse/KAFKA-15818 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-15836) KafkaConsumer subscribe to multiple topics does not respect max.poll.records

2023-11-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15836: -- Summary: KafkaConsumer subscribe to multiple topics does not respect max.poll.records Key: KAFKA-15836 URL: https://issues.apache.org/jira/browse/KAFKA-15836 Project

  1   2   >