Jira ID : Vinoth96
Regards,
Vinoth
d from kafka but
rather be placed differently at a different offset.
Regards,
Vinoth
based on
kip-320 which mentions committed data might be lost when unclean leader
election is enabled.
Regards,
Vinoth
On Mon, 16 Jan 2023 at 10:37, Luke Chen wrote:
> Hi Vinoth,
>
> I'm wondering what's the use case or pain point you're trying to resolve?
> Like y
I am interested in picking up KAFKA-7149
Can I be added to the list? My jira id : vinoth
+1 (non-binding).
On Fri, Sep 6, 2019 at 12:46 AM Bruno Cadonna wrote:
> +1 (non-binding)
>
> On Fri, Sep 6, 2019 at 12:32 AM Guozhang Wang wrote:
> >
> > +1 (binding).
> >
> > On Thu, Sep 5, 2019 at 2:47 PM John Roesler wrote:
> >
> > > Hello, all,
> > >
> > > After a great discussion, I'd li
introduce a configuration to control this. Some
users may prefer errors to stale data. Can we also add it to the KIP?
Thanks
Vinoth
On Sun, Oct 13, 2019 at 3:31 PM Navinder Brar
wrote:
> Hi,
> Starting a discussion on the KIP to Allow state stores to serve stale
> reads during rebalance
by reads provides best
semantics.
Just my 2c. Would love to see what others think as well.
On Tue, Oct 15, 2019 at 5:34 AM Navinder Brar
wrote:
> Hi Vinoth,
> Thanks for the feedback.
> Can we link the JIRA, discussion thread also to the KIP.>> Added.
> Based on the discussion o
thoughts. Can we also work this into the KIP?
I already implemented A to unblock myself for now. Seems feasible to do.
On Tue, Oct 15, 2019 at 12:21 PM Vinoth Chandar
wrote:
> >>I get your point. But suppose there is a replica which has just become
> active, so in that case replica w
w StreamsConfig is good
enough?
On Wed, Oct 16, 2019 at 8:31 PM Navinder Brar
wrote:
> @Vinoth, I have incorporated a few of the discussions we have had in the
> KIP.
>
> In the current code, t0 and t1 serve queries from Active(Running)
> partition. For case t2, we are planning to r
g
> > to use lag information, is that the cluster would only become aware of
> > new lag information during rebalances. Even in the full expression of
> > KIP-441, this information would stop being propagated when the cluster
> > achieves a balanced task distribution. There wa
t `KafkaStreams#state()` method only returns a handle to
> >> stores of active(processing) tasks. How can a user actually get a handle
> >> to an store of an active(restoring) or standby task for querying? Seems
> >> we should add a new method to get standby handles? Ch
e keep the Metadata API / Assignment,Subscription / and Heartbeat as
> slim as possible.
>
> Side note: I do think that some time soon, we'll have to add a library
> for IQ server/clients. I think that this logic will start to get
> pretty complex.
>
> I hope this thinking
>- There is no need to add new StreamsConfig for implementing this KIP
> >
> >- We will add standbyPartitionsByHost in AssignmentInfo and
> > StreamsMetadataState which would change the existing rebuildMetadata()
> and
> > setPartitionsByHostState()
> >
> >
>
t; separate functions. We already have one which returns streamsMetadata for
> active tasks, and now we can add another one for standbys.
>
>
>
> On Saturday, 26 October, 2019, 03:55:16 am IST, Vinoth Chandar <
> vchan...@confluent.io> wrote:
>
> +1 to Sophie'
ed.
>
> 4. Can we avoid adding the new "lag refresh" config? The lags would
> necessarily be approximate anyway, so adding the config seems to
> increase the operational complexity of the system for little actual
> benefit.
>
> Thanks for the pseudocode, by the way, it
r
> > wrote:
> > >
> > > Thanks John for going through this.
> > >
> > >- +1, makes sense
> > >- +1, no issues there
> > >- Yeah the initial patch I had submitted for K-7149(
> > https://github.com/apache/kafka/pull/
ing, we should be able to convert any taskId
> to any
> partitions.
>
> On Mon, Nov 4, 2019 at 11:17 AM John Roesler wrote:
>
> > Hey Vinoth, thanks for the reply!
> >
> > 3.
> > I get that it's not the main focus of this KIP, but if it's ok, it
&
Ping :) Any thoughts?
On Mon, Nov 4, 2019 at 5:13 PM Vinoth Chandar wrote:
> >> I'm having some trouble wrapping my head around what race conditions
> might occur, other than the fundamentally broken state in which different
> instances are running totally different to
e don't need it (a user might not use IQ
> to begin with), it might be worth to maintain an internal flag
> `updateTimeLagEnabled` that is set to `false` initially and only set to
> `true` on the first call of a user to get standby-metadata.
>
>
> -Matthias
>
>
>
> On 11/4
ation.” If we
> add
> > partition number in this, it doesn’t stay metadata for an instance,
> because
> > now it is storing the partition information for a key being queried. So,
> > having “KeyQueryMetadata” simplifies this as now it contains all the
> > metada
reasonings I was laying out. +1 on revisiting rejected alternatives.
Please make the follow up changes
On Wed, Nov 13, 2019 at 9:12 PM Matthias J. Sax
wrote:
> Thanks for the summary Vinoth!
>
> I buy the overall argument. Couple of clarification questions:
>
>
> 10. Why do
do you agree? We can then begin the vote.
On Thu, Nov 14, 2019 at 9:03 AM Navinder Brar
wrote:
> I agree with Vinoth and John on having "allLocalStoreOffsetLags()", all
> actives don't have 0 lag, as during restoring state the active might have
> some lag and one of the
I want know if I can run the bench mark tests for kafka on a single laptop.
If not is there anything available for open source contributors to run
thier tests that is provided by the apache software foundation
Regards,
Vinoth
t; > With 4 binding +1 votes from Guozhang Wang, Matthias J. Sax, Bill Bejeck,
> > and John Roesler, the vote passes.
> > Thanks Guozhang, Matthias, Bill, John, Sophie for the healthy
> discussions and Vinoth for all the help on this KIP.
> > Best,
> > Navinder
>
+1 on merging `StreamsNotRunningException` and
`StateStoreNotAvailableException`, both exceptions are fatal anyway. IMO its
best to have these exceptions be about the state store (and not streams state),
to easier understanding.
Additionally, KIP-535 allows for querying of state stores in reba
y
> want to query stall state during recovery or not).
>
> It's unclear to me though atm, how a user would opt-in/opt-out and what
> the default should be (maybe better to throw the exception by default to
> have strong consistency guarantees by default?)
>
>
> -
Chiming in a bit late here..
+1 This is a very valid improvement. Avoiding doing gets on irrelevant
partitions will improve performance and efficiency for IQs.
As an incremental improvement to the current APIs, adding an option to
filter out based on partitions makes sense
On Mon, Jan 20,
[
https://issues.apache.org/jira/browse/KAFKA-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth resolved KAFKA-10657.
Resolution: Resolved
Has been fixed in https://issues.apache.org/jira/browse/KAFKA-10525
> Incorpor
Vinoth Rajasekar created KAFKA-7820:
---
Summary: distinct count kafka streams api
Key: KAFKA-7820
URL: https://issues.apache.org/jira/browse/KAFKA-7820
Project: Kafka
Issue Type: New Feature
Vinoth Chandar created KAFKA-8799:
-
Summary: Support ability to pass global user data to consumers
during Assignment
Key: KAFKA-8799
URL: https://issues.apache.org/jira/browse/KAFKA-8799
Project
Vinoth Chandar created KAFKA-8810:
-
Summary: Add mechanism to detect topology mismatch between streams
instances
Key: KAFKA-8810
URL: https://issues.apache.org/jira/browse/KAFKA-8810
Project: Kafka
[
https://issues.apache.org/jira/browse/KAFKA-8831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar resolved KAFKA-8831.
---
Resolution: Not A Problem
I will think about better logging and put up a patch. Closing this
Vinoth Chandar created KAFKA-8839:
-
Summary: Improve logging in Kafka Streams around debugging task
lifecycle
Key: KAFKA-8839
URL: https://issues.apache.org/jira/browse/KAFKA-8839
Project: Kafka
Vinoth Chandar created KAFKA-8870:
-
Summary: Prevent dirty reads of Streams state store from
Interactive queries
Key: KAFKA-8870
URL: https://issues.apache.org/jira/browse/KAFKA-8870
Project: Kafka
Vinoth Chandar created KAFKA-8913:
-
Summary: Document topic based configs & ISR settings for Streams
apps
Key: KAFKA-8913
URL: https://issues.apache.org/jira/browse/KAFKA-8913
Project: K
[
https://issues.apache.org/jira/browse/KAFKA-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar resolved KAFKA-8913.
---
Resolution: Fixed
> Document topic based configs & ISR settings for Strea
[
https://issues.apache.org/jira/browse/KAFKA-8839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar resolved KAFKA-8839.
---
Resolution: Fixed
Closing since the PR has been merged
> Improve logging in Kafka Stre
Vinoth Chandar created KAFKA-8994:
-
Summary: Streams should expose standby replication information &
allow stale reads of state store
Key: KAFKA-8994
URL: https://issues.apache.org/jira/browse/KAFKA-
[
https://issues.apache.org/jira/browse/KAFKA-8994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar resolved KAFKA-8994.
---
Resolution: Fixed
> Streams should expose standby replication information & allo
[
https://issues.apache.org/jira/browse/KAFKA-8994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar reopened KAFKA-8994:
---
> Streams should expose standby replication information & allow stale reads of
> s
[
https://issues.apache.org/jira/browse/KAFKA-8994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar resolved KAFKA-8994.
---
Resolution: Duplicate
> Streams should expose standby replication information & allo
Vinoth Chandar created KAFKA-10131:
--
Summary: Minimize use of --zookeeper flag in ducktape tests
Key: KAFKA-10131
URL: https://issues.apache.org/jira/browse/KAFKA-10131
Project: Kafka
Issue
Vinoth Chandar created KAFKA-10138:
--
Summary: Prefer --bootstrap-server for reassign_partitions command
in ducktape tests
Key: KAFKA-10138
URL: https://issues.apache.org/jira/browse/KAFKA-10138
Vinoth Chandar created KAFKA-10174:
--
Summary: Prefer --bootstrap-server ducktape tests using
kafka_configs.sh
Key: KAFKA-10174
URL: https://issues.apache.org/jira/browse/KAFKA-10174
Project: Kafka
[
https://issues.apache.org/jira/browse/KAFKA-10138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar resolved KAFKA-10138.
Resolution: Fixed
> Prefer --bootstrap-server for reassign_partitions command in duckt
Vinoth Chandar created KAFKA-10213:
--
Summary: Prefer --bootstrap-server in ducktape tests for Kafka
clients
Key: KAFKA-10213
URL: https://issues.apache.org/jira/browse/KAFKA-10213
Project: Kafka
Vinoth Chandar created KAFKA-10258:
--
Summary: Get rid of use_zk_connection flag in kafka.py public
methods
Key: KAFKA-10258
URL: https://issues.apache.org/jira/browse/KAFKA-10258
Project: Kafka
Vinoth Chandar created KAFKA-10071:
--
Summary: TopicCommand tool should make more efficient metadata
calls to Kafka Servers
Key: KAFKA-10071
URL: https://issues.apache.org/jira/browse/KAFKA-10071
Vinoth Chandar created KAFKA-9428:
-
Summary: Expose standby information in KafkaStreams via
queryMetadataForKey API
Key: KAFKA-9428
URL: https://issues.apache.org/jira/browse/KAFKA-9428
Project
Vinoth Chandar created KAFKA-9429:
-
Summary: Allow ability to control whether stale reads out of state
stores are desirable
Key: KAFKA-9429
URL: https://issues.apache.org/jira/browse/KAFKA-9429
Vinoth Chandar created KAFKA-9430:
-
Summary: Tighten up lag estimates when source topic optimization
is on
Key: KAFKA-9430
URL: https://issues.apache.org/jira/browse/KAFKA-9430
Project: Kafka
Vinoth Chandar created KAFKA-9431:
-
Summary: Expose API in KafkaStreams to fetch all local offset lags
Key: KAFKA-9431
URL: https://issues.apache.org/jira/browse/KAFKA-9431
Project: Kafka
[
https://issues.apache.org/jira/browse/KAFKA-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar resolved KAFKA-9512.
---
Resolution: Fixed
Closing since the PR is now landed
> Flaky T
Vinoth Rengarajan created KAFKA-14778:
-
Summary: Kafka Streams 2.7.1 to 3.3.1 rolling upgrade with static
membership triggers a rebalance
Key: KAFKA-14778
URL: https://issues.apache.org/jira/browse/KAFKA
[
https://issues.apache.org/jira/browse/KAFKA-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002747#comment-16002747
]
Vinoth Rajasekar commented on KAFKA-4273:
-
we have a use case where we need t
Vinoth Chandar created KAFKA-2580:
-
Summary: Kafka Broker keeps file handles open for all log files
(even if its not written to/read from)
Key: KAFKA-2580
URL: https://issues.apache.org/jira/browse/KAFKA-2580
[
https://issues.apache.org/jira/browse/KAFKA-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908298#comment-14908298
]
Vinoth Chandar commented on KAFKA-2580:
---
More context on how we determined
[
https://issues.apache.org/jira/browse/KAFKA-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908910#comment-14908910
]
Vinoth Chandar commented on KAFKA-2580:
---
Thanks for jumping in [~guozhang]
[
https://issues.apache.org/jira/browse/KAFKA-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941910#comment-14941910
]
Vinoth Chandar commented on KAFKA-2580:
---
A LRU file handle is something
[
https://issues.apache.org/jira/browse/KAFKA-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963574#comment-14963574
]
Vinoth Chandar commented on KAFKA-2580:
---
[~jjkoshy] Good point. if I unders
[
https://issues.apache.org/jira/browse/KAFKA-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984243#comment-14984243
]
Vinoth Chandar commented on KAFKA-2580:
---
Based on this, looks like we can c
[
https://issues.apache.org/jira/browse/KAFKA-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985707#comment-14985707
]
Vinoth Chandar commented on KAFKA-2580:
---
[~jkreps] ah ok. point taken :)
>
62 matches
Mail list logo
