Question Kafka Reassign partitions tool

2017-12-22 Thread Sagar
s, then is there a way I can still do this operation by getting leader for a partition first, then adding it to the replica list and then building the json dynamically? Thanks! Sagar.

Re: Question Kafka Reassign partitions tool

2017-12-22 Thread Sagar
is stored in zookeeper? If yes, then can I probably use that get the current json and then build the json which can be fed to the tool. I tried to search using the zk shell but couldn't find... Sagar. On Fri, Dec 22, 2017 at 7:45 PM, Todd Palino wrote: > Preferred replica election is n

Kafka Consumer manual partition assignment

2017-12-26 Thread Sagar
e are using AWS ECS for running our consumers so target group would spin up a new consumer based upon health checks. Any examples + guidelines around this would be highly appreciated. Thanks! Sagar.

Re: Question Kafka Reassign partitions tool

2017-12-26 Thread Sagar
the first entry in that list, basically to keep the preferred replica logic intact. Thanks for the help. Thanks! Sagar. On Fri, Dec 22, 2017 at 10:08 PM, Todd Palino wrote: > Yes, the replicas are stored in Zookeeper, so you can iterate over the > information there to build a view of the c

Re: Kafka Consumer manual partition assignment

2018-01-02 Thread Sagar
Hi, Any help here would be highly appreciated :) Sagar. On Wed, 27 Dec 2017 at 07:53, Sagar wrote: > We have a use case where in we want to assign partitions manually for a > set of topics to allow fine grained control of the records we are fetching. > > Basically what we a

Question Regarding seek() in KafkaConsumer

2018-01-03 Thread Sagar
this way? If I want to reliably retry then what patterns should be used? Couple of other approaches I read in Kafka definitive gide were to push the retriable records to a buffer or to a separate topic. So, are those the correct ways of retrying and not this? Thanks! Sagar.

KAFKA-8238

2019-04-26 Thread Sagar
Hi all, I would like to start contributing to the Kafka project. I looked at the newbie tickets and thought I could start with this:(?) https://issues.apache.org/jira/browse/KAFKA-8238 Or if there's anything else that I can look at, please let me know. Sagar.

Re: Kafka Connect

2019-06-04 Thread Sagar
Hi meirkhan, You can give debezium a shot. It has a connector for mongodb On Tue, 4 Jun 2019 at 6:56 PM, wrote: > Hello Kafka! > > Here is my case. I need the Kafka Connector to Mongodb which imitates JDBC > Source Connector. > So that data is loaded to Kafka by periodically executing a query.

Need Help regarding Jenkins buil

2016-10-23 Thread Sagar
? How can I have this PR being reviewed by someone? Thanks! Sagar.

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-06-09 Thread Sagar
emails. If it makes sense, I can modify my PR to go along these lines. Please let me know what you think. Lastly, I didn't understand this line of yours: *It might help if there are other typed key/value stores to compare APIs with.* Thanks! Sagar. On Thu, Jun 4, 2020 at 6:03 AM John Ro

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-06-09 Thread Sagar
is.io/commands/scan#the-match-option Typically Redis works with Binary Safe strings so the prefix key type and the actual keys are of the same type. Thanks! Sagar. On Wed, Jun 10, 2020 at 1:41 AM John Roesler wrote: > Hi Sagar, > > Thanks for the reply. I agree that your UUID example i

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-06-28 Thread Sagar
chance, it would be great if you can provide some feedback on this. Thanks! Sagar. On Wed, Jun 10, 2020 at 9:21 AM Sagar wrote: > Hi John, > > You rightly pointed out, the devil is in the detail :). I will start with > the implementation to get a sense. > > Here are my thought

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-06-28 Thread Sagar
should I move it to ReadOnlyKeyValueStore where even the range query resides? Regarding the 2 notes on UnsupportedOperationException and changing the name to prefixScan, i will incorporate both of them into the KIP. Thanks! Sagar. On Sun, Jun 28, 2020 at 11:55 PM John Roesler wrote: > W

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-07-04 Thread Sagar
Hi John, I have updated the KIP with all the new changes we discussed in this discussion thread. https://cwiki.apache.org/confluence/display/KAFKA/KIP-614%3A+Add+Prefix+Scan+support+for+State+Stores Request you to go through the same. Thanks! Sagar. On Tue, Jun 30, 2020 at 8:09 AM John

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-07-05 Thread Sagar
Hi John, Thank you! Question on the comment, where should I add the default implementation? I guess that needs to be added in the Proposal Section of the kIP. Thanks! Sagar. On Sat, Jul 4, 2020 at 11:46 PM John Roesler wrote: > Thanks Sagar, > > That looks good to me! The only mino

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-07-06 Thread Sagar
Hi John, Thanks, I have updated the KIP. Thanks! Sagar. On Mon, Jul 6, 2020 at 12:00 AM John Roesler wrote: > Hi Sagar, > > Sorry for the ambiguity. You could just mention it in the Public > Interfaces section. Or, if you want to be more specific, you can show it in > the me

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-07-11 Thread Sagar
Thanks John, Sorry I’m new to this process. 😅 does it mean I start a voting email? Pardon my ignorance. Sagar. On Sat, 11 Jul 2020 at 8:06 PM, John Roesler wrote: > Hi Sagar, > > Thanks for the update. As far as I’m concerned, I’m ready to vote now. > > Thanks, > John

[VOTE] KIP-614: Add Prefix Scan support for State Stores

2020-07-12 Thread Sagar
can+support+for+State+Stores> Thanks! Sagar.

Re: [VOTE] KIP-614: Add Prefix Scan support for State Stores

2020-07-19 Thread Sagar
Hi All, Bumping this thread to see if there are any feedbacks. Thanks! Sagar. On Tue, Jul 14, 2020 at 9:49 AM John Roesler wrote: > Thanks for the KIP, Sagar! > > I’m +1 (binding) > > -John > > On Sun, Jul 12, 2020, at 02:05, Sagar wrote: > > Hi All, > > >

Re: [VOTE] KIP-614: Add Prefix Scan support for State Stores

2020-08-02 Thread Sagar
Hi All, Just thought of bumping this voting thread again to see if we can form any consensus around this. Thanks! Sagar. On Mon, Jul 20, 2020 at 4:21 AM Adam Bellemare wrote: > LGTM > +1 non-binding > > On Sun, Jul 19, 2020 at 4:13 AM Sagar wrote: > > > Hi All, > &

Re: [VOTE] KIP-614: Add Prefix Scan support for State Stores

2020-09-01 Thread Sagar
Hi All, Bumping the thread again ! Thanks! Sagar. On Wed, Aug 5, 2020 at 12:08 AM Sophie Blee-Goldman wrote: > Thanks Sagar! +1 (non-binding) > > Sophie > > On Sun, Aug 2, 2020 at 11:37 PM Sagar wrote: > > > Hi All, > > > > Just thought of bumping this

Re: [VOTE] KIP-614: Add Prefix Scan support for State Stores

2020-09-02 Thread Sagar
Thanks All! I see 3 binding +1 votes and 2 non-binding +1s. Does it mean this KIP has gained a lazy majority? Thanks! Sagar. On Thu, Sep 3, 2020 at 6:51 AM Guozhang Wang wrote: > Thanks for the KIP Sagar. I'm +1 (binding) too. > > > Guozhang > > On Tue, Sep 1, 2020

Re: [VOTE] KIP-614: Add Prefix Scan support for State Stores

2020-09-03 Thread Sagar
Hi John, Thank you! I have marked the KIP as Accepted :) Regarding the point on InMemoryKeyValueStore, in the PR I had added the implementation for InMemoryKeyValueStore as well. I hadn't mentioned about it in the KIP which I have done now as you suggested. Thanks! Sagar. On Thu, Sep 3,

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

2023-07-25 Thread Sagar
Compatibility section, you have mentioned backward comparable. I believe it should be *backward compatible change.* I don't have other comments. Post this, probably someone else who has more context on Clients can also chime in on this before we can move this to Voting. Thanks! Sagar. On Sat, J

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

2023-07-27 Thread Sagar
e (similar to the effect of setting *partitioner.ignore.keys *config to false) and document it appropriately? Let me know what you think. Thanks! Sagar. On Thu, Jul 27, 2023 at 9:41 PM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Hi Jack, > Thanks for the KIP.

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-07-28 Thread Sagar
times, keeps things simpler to reason about. Let me know how things look now. If it's all looking ok, I would go ahead and create a Vote thread for the same. Thanks! Sagar. On Tue, Jul 25, 2023 at 5:15 PM Yash Mayya wrote: > Hi Sagar, > > Thanks for the updates. I had a few more fo

Re: [DISCUSS] KIP-953: partition method to be overloaded to accept headers as well.

2023-07-29 Thread Sagar
seem achievable by simply attaching the partition number in the ProducerRecord. I guess if we can't find any further examples which strengthen the case of this partitioner, it might be harder to justify adding it. Thanks! Sagar. On Fri, Jul 28, 2023 at 2:05 PM Andrew Schofield < andre

[VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-02 Thread Sagar
Hi All, Calling a Vote on KIP-910 [1]. I feel we have converged to a reasonable design. Ofcourse I am open to any feedback/suggestions and would address them. Thanks! Sagar.

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-02 Thread Sagar
Attaching the KIP link for reference: https://cwiki.apache.org/confluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Source+Connectors+without+producing+records Thanks! Sagar. On Wed, Aug 2, 2023 at 4:37 PM Sagar wrote: > Hi All, > > Calling a Vote on KIP-910 [1]. I fee

Re: [VOTE] KIP-953: partition method to be overloaded to accept headers as well.

2023-08-11 Thread Sagar
Hey jack , +1 (non binding) Sagar. On Sat, 12 Aug 2023 at 8:04 AM, Jack Tomy wrote: > Hey everyone, > > Please consider this as a gentle reminder. > > On Mon, Aug 7, 2023 at 5:55 PM Jack Tomy wrote: > > > Hey everyone. > > > > I would like to call for a

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-17 Thread Sagar
Hi All, Bumping the voting thread again. Thanks! Sagar. On Wed, Aug 2, 2023 at 4:43 PM Sagar wrote: > Attaching the KIP link for reference: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-910%3A+Update+Source+offsets+for+Source+Connectors+without+producing+records > > Tha

Re: Re: [DISCUSS] KIP-943: Add independent "offset.storage.segment.bytes" for connect-distributed.properties

2023-08-17 Thread Sagar
of a Point of view(or even a potential fix) on this before we go about implementing this change. WDYT? Thanks! Sagar. On Mon, Aug 14, 2023 at 6:09 PM hudeqi <16120...@bjtu.edu.cn> wrote: > bump this discuss thread. > > best, > hudeqi > > "hudeqi" <16120...

Re: [DISCUSS] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

2023-08-23 Thread Sagar
Thanks Yash. LGTM Thanks! Sagar. On Tue, Aug 22, 2023 at 6:04 PM Chris Egerton wrote: > Hi Yash, > > Thanks for driving this, and for putting out a well-written KIP. LGTM! > > Cheers, > > Chris > > On Tue, Aug 22, 2023 at 6:13 AM Yash Mayya wrote: > > >

Re: [VOTE] KIP-953: partition method to be overloaded to accept headers as well.

2023-08-27 Thread Sagar
existing partition method expects serialized keys and values while ProducerRecord doesn't have access to those (It directly operates on K, V). Thanks! Sagar. On Sun, Aug 27, 2023 at 8:51 AM Ismael Juma wrote: > A more concrete question: did we consider having the method `partition` >

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-08-28 Thread Sagar
n such thresholds are breached. Even here we could argue that since it's sending the partition/offsets it can do the tracking on it's own, but IMO that is too much work given that the information is already available via offsets to be committed. Thanks! Sagar.

Disabling Test: org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated()

2023-08-28 Thread Sagar
know if disabling it doesn't sound like a good idea. Thanks! Sagar.

Re: Disabling Test: org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated()

2023-08-28 Thread Sagar
Hey Greg, Aah ok, I wasn't aware there existed a JIRA for this already. I did see your attempt to fix this but it seems to be failing still. Sagar. On Mon, Aug 28, 2023 at 10:30 PM Greg Harris wrote: > Hey Sagar, > > The JIRA for this flaky test is here: > https://issue

Re: [VOTE] KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

2023-08-30 Thread Sagar
+1 (non - binding). Thanks ! Sagar. On Wed, 30 Aug 2023 at 11:09 PM, Chris Egerton wrote: > +1 (binding), thanks Yash! > > On Wed, Aug 30, 2023 at 1:34 PM Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > > > Thanks for the KIP. Looks good to

Re: KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-09-05 Thread Sagar
ld be similar to the default case. 4) I had the same question as Yash regarding persistent cluster-wide logging level. I think you have explained it well and we can skip it for now. Thanks! Sagar. On Tue, Sep 5, 2023 at 8:49 PM Chris Egerton wrote: > Hi all, > > Thank you so much for the ge

Re: [VOTE] KIP-953: partition method to be overloaded to accept headers as well.

2023-09-05 Thread Sagar
Also, checking out the AdminClient APIs examples provided by Ismael will give you more context. Let me know what you think. Thanks! Sagar. On Thu, Aug 31, 2023 at 12:49 PM Jack Tomy wrote: > Hey everyone, > > As I see devs favouring the current style of implementation, and that is > inl

Re: KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-09-05 Thread Sagar
y achievable by invoking scope=worker endpoint n times to reflect on n workers so maybe not a necessary change. But this could be useful on a large cluster. Do you think this is worth listing in the Future Work section? It's not important so can be ignored as well. Thanks! Sagar. On Wed, Se

Re: [DISCUSS] KIP-910: Update Source offsets for Source Connectors without producing records

2023-09-06 Thread Sagar
ne records. Thanks Yash for closing out on this one! Hopefully all open questions have now been addressed. Thanks! Sagar. On Tue, Aug 29, 2023 at 3:33 PM Yash Mayya wrote: > Hi Sagar, > > > The size of offsets topic can be controlled by > > setting appropriate topic retent

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-09-06 Thread Sagar
Hi All, Based on the latest discussion thread, it appears as if all open questions have been answered. Hopefully now we are in a state where we can close out on the Voting process. Thanks everyone for the great feedback. Thanks! Sagar. On Fri, Aug 18, 2023 at 9:00 AM Sagar wrote: > Hi

Re: KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-09-10 Thread Sagar
lated activities per worker on a subset of them using finer grained logs. I am not sure if it's a strong enough motivation but as I said it just felt like a useful extension. I will leave it to you if you want to add it or not (I am ok either way). Thanks! Sagar. On Thu, Sep 7, 2023 at 9:2

Re: [VOTE] 3.6.0 RC0

2023-09-19 Thread Sagar
to this). Would like to hear others' thoughts as well. Thanks! Sagar. On Tue, Sep 19, 2023 at 3:14 PM Satish Duggana wrote: > Hi Greg, > Thanks for reporting the KafkaConnect issue. I replied to this issue > on "Apache Kafka 3.6.0 release" email thread and on > htt

Re: [ANNOUNCE] New committer: Yash Mayya

2023-09-21 Thread Sagar
Congrats Yash ! On Thu, 21 Sep 2023 at 9:38 PM, Ashwin wrote: > Awesome ! Congratulations Yash !! > > On Thu, Sep 21, 2023 at 9:25 PM Edoardo Comar > wrote: > > > Congratulations Yash > > > > On Thu, 21 Sept 2023 at 16:28, Bruno Cadonna wrote: > > > > > > Hi all, > > > > > > The PMC of Apache K

Re: [ANNOUNCE] New committer: Lucas Brutschy

2023-09-21 Thread Sagar
Congrats Lucas ! On Thu, 21 Sep 2023 at 9:15 PM, Bruno Cadonna wrote: > Hi all, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer > Lucas Brutschy. > > Lucas' major contributions are around Kafka Streams. > > Lucas' significantly contributed to the state updater > (https:/

Re: [DISCUSS] KIP-987: Connect Static Assignments

2023-10-18 Thread Sagar
led rebalance delays? Maybe this has been explained in the KIP but it wasn't clear to me when I read it. Thanks! Sagar. On Tue, Oct 10, 2023 at 11:22 PM Greg Harris wrote: > Hi Mickael, > > I'm not Chris but I hope I can still respond to your questions :) > > 1a. This s

Re: Security for Kafka

2023-10-18 Thread Sagar
Hey Walchester, There's a confluent community slack Workspace having a #security channel where you can post your question. Also, have you filed a bug in AK JIRA which can also help you get the traction of some of the community members who have expertise in this area. Thanks! Sagar. On Thu

Re: [DISCUSS] KIP-987: Connect Static Assignments

2023-10-27 Thread Sagar
nfig values suggested above should cover all cases, but I still do agree to the fact that setting the selector config value at a task level is a challenge. Thanks! Sagar. On Fri, Oct 20, 2023 at 10:43 PM Greg Harris wrote: > Hey Hector, > > That's a cool idea for the ConnectA

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-11 Thread Sagar
ld get confusing at times especially if the flakey test involved is on an unrelated part of code (like a flaky Connect test on Group Coordinator or Streams). Thanks! Sagar. On Sat, Nov 11, 2023 at 3:31 PM David Jacot wrote: > Hi all, > > The state of our CI worries me a lot. Just t

Re: [DISCUSS] Should we continue to merge without a green build? No!

2023-11-13 Thread Sagar
as a first time contributor, it might be difficult. newbie++ IMO covers that aspect. [1] https://issues.apache.org/jira/browse/KAFKA-15406?jql=project%20%3D%20KAFKA%20AND%20labels%20%3D%20%22newbie%2B%2B%22 Let me know what you think. Thanks! Sagar. On Mon, Nov 13, 2023 at 9:11 PM Divij V

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2023-11-15 Thread Sagar
Hey all, Bumping this vote thread again after quite a while. Thanks! Sagar. On Wed, Sep 6, 2023 at 3:58 PM Sagar wrote: > Hi All, > > Based on the latest discussion thread, it appears as if all open questions > have been answered. > > Hopefully now we are in a state where w

[DISCUSS] KIP-1083: Increase default value of task.shutdown.graceful.timeout.ms in Connect

2024-08-14 Thread Sagar
Hey everyone, I created KIP-1083 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-1083%3A+Increase+default+value+of+task.shutdown.graceful.timeout.ms+in+Connect> to increase the default value of task.shutdown.graceful.timeout.ms config in Connect. Please review. Thanks! Sagar.

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2023-12-27 Thread Sagar
Congrats Divij! Absolutely well deserved ! Thanks! Sagar. On Wed, Dec 27, 2023 at 5:15 PM Luke Chen wrote: > Hi, Everyone, > > Divij has been a Kafka committer since June, 2023. He has remained very > active and instructive in the community since becoming a committer. It's

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2024-02-02 Thread Sagar
Thanks Yash! I am hoping to have this released in 3.8 so it would be good to get the remaining 2 votes. Thanks! Sagar. On Tue, Jan 30, 2024 at 3:18 PM Yash Mayya wrote: > Hi Sagar, > > Thanks for the KIP and apologies for the extremely long delay here! I think > we could

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Sagar
Congrats Christo! Sagar. On Tue, 26 Mar 2024 at 6:04 PM, Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Congrats Christo!! > > On Tue 26. Mar 2024 at 14.33, Apoorv Mittal > wrote: > > > Congrats Christo! > > > > Regards, > > Ap

Re: [VOTE] KIP-910: Update Source offsets for Source Connectors without producing records

2024-04-25 Thread Sagar
Hey All, Bumping the vote thread after a long time! Thanks! Sagar. On Fri, Feb 2, 2024 at 4:24 PM Sagar wrote: > Thanks Yash! > > I am hoping to have this released in 3.8 so it would be good to get the > remaining 2 votes. > > Thanks! > Sagar. > > > On Tue, Jan

[DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-05-11 Thread Sagar
Hi All, I would like to start a discussion on the KIP that I created below to add prefix scan support in State Stores: https://cwiki.apache.org/confluence/display/KAFKA/KIP-614%3A+Add+Prefix+Scan+support+for+State+Stores Thanks! Sagar.

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-05-14 Thread Sagar
So, pardon any incorrect/illogical statements plz! @Sophie, Thanks for bringing that point up! I have mentioned about that PR in the KIP under a section called Other considerations. Nonetheless, thanks for pointing it out! Thanks! Sagar. On Thu, May 14, 2020 at 5:17 AM Sophie Blee-Goldman wro

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-05-26 Thread Sagar
sponse was not at all discouraging. On the contrary it was really insightful and it's always good to learn/discover new things :) Thanks! Sagar. On Fri, May 15, 2020 at 7:37 AM John Roesler wrote: > Hi, Sagar! > > Thanks for this KIP. I'm sorry it took me so long to reply. I

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-05-31 Thread Sagar
type definition of types I will try and think about some alternatives and share if I get any. Thanks! Sagar. On Sun, May 31, 2020 at 1:55 AM John Roesler wrote: > Hi Sagar, > > Thanks for the response. Your use case makes sense to me; I figured it > must be something like th

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

2020-06-01 Thread Sagar
d for searching . Thanks! Sagar. On Mon, Jun 1, 2020 at 11:42 AM Sagar wrote: > Hi John, > > Thank you. I think it makes sense to modify the KIP to add the > prefixScan() as part of the existing interfaces and add the new mixin > behaviour as Rejected alternatives. I am not very a

Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol

2022-08-12 Thread Sagar
Thank you Guozhang/David for the feedback. Looks like there's agreement on using separate APIs for Connect. I would revisit the doc and see what changes are to be made. Thanks! Sagar. On Tue, Aug 9, 2022 at 7:11 PM David Jacot wrote: > Hi Sagar, > > Thanks for the feedback an

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-08-12 Thread Sagar
! Sagar. On Sat, Aug 13, 2022 at 7:35 AM John Roesler wrote: > Thanks, Sagar! > > I’m +1 (binding) > > Can you add a short explanation to each rejected alternative? I was > wondering why we wouldn’t provide an overloaded to()/addSink() (the first > rejected alternative), and I ha

Re: [DISCUSS] KIP-844: Transactional State Stores

2022-08-17 Thread Sagar
this, I still wanted to ask that in Kafka Streams, from my limited knowledge, if EOS is enabled, it would return only committed data. So, I'm still curious about the choice of going to the secondary store first. Maybe there's something fundamental that I am missing. Thanks for the KIP ag

Re: [DISCUSS] KIP-844: Transactional State Stores

2022-08-17 Thread Sagar
querying we should be able to figure out which store to hit. But that seems to be a case of over optimisation upfront. Either way, bloom filters should be able to help us out as you pointed out. Thanks! Sagar. On Wed, Aug 17, 2022 at 6:05 PM Alexander Sorokoumov wrote: > Hey Sagar, >

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-08-18 Thread Sagar
Hello Sophie, Thanks for your feedback. I have made all the suggested changes. One note, on how users can accomplish this in today's world , I have made up this example and have never tried myself before. But I am assuming it will work. Let me know what you think. Thanks! Sagar. On Thu

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-08-19 Thread Sagar
lways filter the records out initially. Let me know if this looks ok? Thanks! Sagar. On Fri, Aug 19, 2022 at 10:17 AM Sophie Blee-Goldman wrote: > Thanks Sagar -- one thing I'm still confused about, and sorry to keep > pushing on this, but the example > you gave for how this wor

Re: [DISCUSS] KIP-848: The Next Generation of the Consumer Rebalance Protocol

2022-08-22 Thread Sagar
uld also depend on the group coordinator and whether they support the new rebalance protocol or not. I have tried to incorporate this aspect in the draft page. Thanks! Sagar. On Mon, Aug 22, 2022 at 1:00 PM Luke Chen wrote: > Hi David, > > Thanks for the update. > > Some more q

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-08-24 Thread Sagar
optional and an empty list have totally different meanings which could get confusing. Let me know what you think. Thanks! Sagar. On Wed, Aug 24, 2022 at 7:30 PM Matthew Benedict de Detrich wrote: > I also concur with this, having an Optional in the type makes it very > clear what’s going

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-08-30 Thread Sagar
on of partitions method based on the great catch from Chris! Let me know if it looks fine now. Thanks! Sagar. On Tue, Aug 30, 2022 at 3:00 PM Bruno Cadonna wrote: > Hi, > > I am favour of discarding the sugar for broadcasting and leave the > broadcasting to the implementation as Chris sugge

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-08-31 Thread Sagar
tly about the partition number check. I wanted to avoid the throwing of exception so I thought adding it might be a useful feature. But as you pointed out, if it can break backwards compatibility, it's better to remove it. Thanks! Sagar. On Tue, Aug 30, 2022 at 6:32 PM Chris Egerton wrote:

Re: [DISCUSS] KIP-864: Add End-To-End Latency Metrics to Connectors

2022-09-01 Thread Sagar
ecords to external sink. So, I would assume the 2 can't be added as is to compute the e2e latency. Maybe I am missing something here. Could you plz clarify this. Thanks! Sagar. On Tue, Aug 30, 2022 at 8:43 PM Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Hi all, &g

Re: [DISCUSS] KIP-864: Add End-To-End Latency Metrics to Connectors

2022-09-02 Thread Sagar
cult to track the final leg via the framework. Probably users can track the metrics at their end to figure that out. Do you think that makes sense? Thanks! Sagar. On Thu, Sep 1, 2022 at 11:40 PM Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Hi Sagar and Yash,

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-09-02 Thread Sagar
Hello Bruno/Chris, Since these are the last set of changes(I am assuming haha), it would be great if you could review the 2 options from above so that we can close the voting. Of course I am happy to incorporate any other requisite changes. Thanks! Sagar. On Wed, Aug 31, 2022 at 10:07 PM Sagar

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-09-10 Thread Sagar
Hi Bruno, Thanks, I think these changes make sense to me. I have updated the KIP accordingly. Thanks! Sagar. On Wed, Sep 7, 2022 at 2:16 PM Bruno Cadonna wrote: > Hi Sagar, > > I would not drop the support for dropping records. I would also not > return null from partitions

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-09-12 Thread Sagar
Thanks Bruno, Marking this as accepted. Thanks everyone for their comments/feedback. Thanks! Sagar. On Mon, Sep 12, 2022 at 1:53 PM Bruno Cadonna wrote: > Hi Sagar, > > Thanks for the update and the PR! > > +1 (binding) > > Best, > Bruno > > On 10.09.22 18

Re: [ANNOUNCE] New committer: Deng Ziming

2022-10-10 Thread Sagar
Congratulations ziming! On Tue, 11 Oct 2022 at 4:16 AM, Bill Bejeck wrote: > Congrats Ziming! > > Regards, > Bill > > On Mon, Oct 10, 2022 at 5:32 PM Ismael Juma wrote: > > > Congratulations Ziming! > > > > Ismael > > > > On Mon, Oct 10, 2022 at 9:30 AM Jason Gustafson > > > > > wrote: > > > >

Re: [DISCUSS] KIP-878: Autoscaling for Statically Partitioned Streams

2022-10-27 Thread Sagar
ature could bring in? Thanks! Sagar. On Wed, Oct 26, 2022 at 2:15 AM Walker Carlson wrote: > Hey Sophie, > > Thanks for the KIP. I think this could be useful for a lot of cases. I also > think that this could cause a lot of confusion. > > Just to make sure we are doing our best

Re: [ANNOUNCE] New Kafka PMC Member: Bruno Cadonna

2022-11-01 Thread Sagar
Congrats Bruno! Sagar. On Wed, Nov 2, 2022 at 7:51 AM deng ziming wrote: > Congrats! > > -- > Ziming > > > On Nov 2, 2022, at 3:36 AM, Guozhang Wang wrote: > > > > Hi everyone, > > > > I'd like to introduce our new Kafka PMC member, Bruno. &g

Re: [DISCUSS] KIP-882: Make Kafka Connect REST API request timeouts configurable

2022-11-03 Thread Sagar
value IMO. Of course I am also open to suggestions here. Thanks! Sagar. On Thu, Nov 3, 2022 at 9:01 PM Chris Egerton wrote: > Hi Yash, > > RE 2: That's a great point about validations already being performed by the > leader. For completeness's sake, I'd like to note tha

Re: [DISCUSS] KIP-882: Make Kafka Connect REST API request timeouts configurable

2022-11-05 Thread Sagar
Hey Yash, Thanks for the explanation. I think it should be fine to delegate the validation directly to the leader. Thanks! Sagar. On Sat, Nov 5, 2022 at 10:42 AM Yash Mayya wrote: > Hi Sagar, > > Thanks for chiming in! > > > Having said that, why does the worker forward to

Re: [DISCUSS] KIP-883: Add delete callback method to Connector API

2022-11-15 Thread Sagar
shutdown or else try a force shutdown. What do you think about that approach? Thanks! Sagar. On Sat, Nov 12, 2022 at 1:53 AM Hector Geraldino (BLOOMBERG/ 919 3RD A) < hgerald...@bloomberg.net> wrote: > Thanks Greg for taking your time to review not just the KIP but also the > PR. > >

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-22 Thread Sagar
! Sagar. On Tue, Nov 22, 2022 at 2:10 PM Ashwin wrote: > Hi Snehasis, > > > IIUC (please correct me if I am wrong here), what you highlighted above, > is > a versioning scheme for a connector config for the same connector (and not > different versions of a connector plugin). &g

Re: [DISCUSS] KIP-889 Versioned State Stores

2022-11-22 Thread Sagar
ing based on new records while *from* is fixed. WDYT? 4) Even I think delete api should be supported. Thanks! Sagar. On Tue, Nov 22, 2022 at 8:02 AM Matthias J. Sax wrote: > Thanks for the KIP Victoria. Very well written! > > > Couple of questions (many might just require to add so

Re: [DISCUSS] KIP-889 Versioned State Stores

2022-11-23 Thread Sagar
it. After going through the link and your KIP, it's a lot clearer to me. 4) I think delete(key) should be sufficient. With delete, we would stlll keep the older versions of the key right? Thanks! Sagar. On Wed, Nov 23, 2022 at 12:17 AM Victoria Xia wrote: > Thanks, Matthias and Sagar, for y

Re: [DISCUSS] KIP-889 Versioned State Stores

2022-12-01 Thread Sagar
Thanks Victoria, I guess an advantage of exposing a method like delete(key, timestamp) could be that from a user's standpoint, it is a single operation and not 2. The equivalent of this method i.e put followed by get is not atomic so exposing it certainly sounds like a good idea. Thanks!

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-06 Thread Sagar
: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211883356 Thanks! Sagar. On Mon, Sep 12, 2022 at 6:43 PM Sagar wrote: > Thanks Bruno, > > Marking this as accepted. > > Thanks everyone for their comments/feedback. > > Thanks! > Sagar. > > On Mon,

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-07 Thread Sagar
as another partitions() method to signify the same. FK-Join also has a similar case, but while reviewing it was felt that FK-Join on it's own is fairly complicated and we don't need this feature right away so the restriction still exists. Thanks! Sagar. On Wed, Dec 7, 2022 at 9:42 PM Matthias

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-08 Thread Sagar
he PR link: https://github.com/apache/kafka/pull/12803. Thanks! Sagar. On Fri, Dec 9, 2022 at 5:15 AM Matthias J. Sax wrote: > Ah. Missed it as it does not have a nice "code block" similar to > `StreamPartitioner` changes. > > I understand the motivation, but I am wonder

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-09 Thread Sagar
is at this point as well. And I would keep in mind the feedback to provide more time in such a scenario. Thanks! Sagar. On Fri, Dec 9, 2022 at 11:41 PM Matthias J. Sax wrote: > It is what it is. > > > we did have internal discussions on this > > We sometimes have the case that a

Re: [ANNOUNCE] New Kafka PMC Member: Luke Chen

2022-12-17 Thread Sagar
Congratulations Luke! Very well deserved! Sagar. On Sun, 18 Dec 2022 at 6:41 AM, Sam Barker wrote: > Congratulations Luke! > > On Sat, 17 Dec 2022 at 08:41, Jun Rao wrote: > > > Hi, Everyone, > > > > Luke Chen has been a Kafka committer since Feb. 9, 2022. He has

Re: [VOTE] KIP-889 Versioned State Stores

2022-12-20 Thread Sagar
Hi Victoria, +1 (non-binding). Thanks! Sagar. On Tue, Dec 20, 2022 at 1:39 PM Bruno Cadonna wrote: > Hi Victoria, > > Thanks for the KIP! > > +1 (binding) > > Best, > Bruno > > On 19.12.22 20:03, Matthias J. Sax wrote: > > +1 (binding) > > &

Re: [VOTE] KIP-837 Allow MultiCasting a Result Record.

2022-12-21 Thread Sagar
stand, KeyQueryMetada exposes only the partition() method and the restriction for single partition is added back for IQ. This has been done based on the points raised by Matthias above. The KIP has been updated accordingly. Thanks! Sagar. On Sat, Dec 10, 2022 at 12:09 AM Sagar wrote: >

Re: [ANNOUNCE] New committer: Walker Carlson

2023-01-17 Thread Sagar
Congratulations Walker! Thanks! Sagar. On Wed, Jan 18, 2023 at 9:32 AM Tom Bentley wrote: > Congratulations! > > On Wed, 18 Jan 2023 at 01:26, John Roesler wrote: > > > Congratulations, Walker! > > -John > > > > On Tue, Jan 17, 2023, at 18:50, Guozhan

Permission to create KIP

2019-12-27 Thread Sagar
Hi, I have done some work on adding prefix scan for state store and would like to create a KIP for the same. Thanks! Sagar.

Re: Permission to create KIP

2019-12-30 Thread Sagar
Hi, Its sagarmeansocean On Sat, Dec 28, 2019 at 12:29 AM Matthias J. Sax wrote: > What is your wiki account id? > > > -Matthias > > On 12/27/19 7:12 AM, Sagar wrote: > > Hi, > > > > I have done some work on adding prefix scan for state store and would >

Re: [VOTE] KIP-811: Add config min.repartition.purge.interval.ms to Kafka Streams

2022-01-16 Thread Sagar
of them is very high as compared to the other. Let me know what you think. Other than that, Im +1 (non-binding). Thanks! Sagar. On Sun, Jan 16, 2022 at 6:17 PM Luke Chen wrote: > Hi Nick, > > Thanks for the KIP! > +1 (non-binding) > > Luke > > On Sat, Jan 15, 2022 at

Re: [VOTE] KIP-770: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-01-22 Thread Sagar
Hi All, There is a small update to the KIP whereby the newly introduced metric *total-bytes *has been renamed to *input-buffer-bytes-total.* Thanks! Sagar. On Wed, Sep 29, 2021 at 9:57 AM Sagar wrote: > We have 3 binding votes: Sophie/Guozhang/Mathias > and 2 non-binding votes: Josep/Lu

  1   2   3   >