Thanks for clarifying. Makes sense to me.
On 11/30/23 8:33 PM, Colt McNealy wrote:
Hi Matthias and everyone—
Some clarification is necessary just for posterity. It turns out that, on a
fresh standby task before we start polling for records, we wouldn't be able
to get the current end offset with
Hi Matthias and everyone—
Some clarification is necessary just for posterity. It turns out that, on a
fresh standby task before we start polling for records, we wouldn't be able
to get the current end offset without a network call. This leaves us three
options:
A) Make it an Optional or use a sen
parameter is somewhat irrelevant to our use case
Sounds like a weird justification to change the KIP. Providing more
information is usually better than less, so it seems it won't hurt to
just keep it (seems useful in general to get the current end offset in
this callback) -- you can always ig
Hello everyone,
We have come to the conclusion, during our work on this KIP's
implementation, that the #onUpdateStart callback's "currentEndOffset"
parameter is somewhat irrelevant to our use case. When this callback is
invoked, I think this value is usually unknown. Our choice to delete this
para
Thanks. SGTM.
On 10/25/23 4:06 PM, Sophie Blee-Goldman wrote:
That all sounds good to me! Thanks for the KIP
On Wed, Oct 25, 2023 at 3:47 PM Colt McNealy wrote:
Hi Sophie, Matthias, Bruno, and Eduwer—
Thanks for your patience as I have been scrambling to catch up after a week
of business tr
That all sounds good to me! Thanks for the KIP
On Wed, Oct 25, 2023 at 3:47 PM Colt McNealy wrote:
> Hi Sophie, Matthias, Bruno, and Eduwer—
>
> Thanks for your patience as I have been scrambling to catch up after a week
> of business travel (and a few days with no time to code). I'd like to tie
Hi Sophie, Matthias, Bruno, and Eduwer—
Thanks for your patience as I have been scrambling to catch up after a week
of business travel (and a few days with no time to code). I'd like to tie
up some loose ends here, but in short, I don't think the KIP document
itself needs any changes (our internal
Just want to checkpoint the current state of this KIP and make sure we're
on track to get it in to 3.7 (we still have a few weeks) -- looks like
there are two remaining open questions, both relating to the
middle/intermediate callback:
1. What to name it: seems like the primary candidates are onB
Forgot one thing:
We could also pass `currentLag()` into `onBachLoaded()` instead of
end-offset.
-Matthias
On 10/20/23 10:56 AM, Matthias J. Sax wrote:
Thanks for digging into this Bruno.
The JavaDoc on the consumer does not say anything specific about
`endOffset` guarantees:
Get the en
Thanks for digging into this Bruno.
The JavaDoc on the consumer does not say anything specific about
`endOffset` guarantees:
Get the end offsets for the given partitions. In the default {@code
read_uncommitted} isolation level, the end
offset is the high watermark (that is, the offset of the
Hi,
Matthias is correct that the end offsets are stored somewhere in the
metadata of the consumer. More precisely, they are stored in the
`TopicPartitionState`. However, I could not find public API on the
consumer other than currentLag() that uses the stored end offsets. If I
understand the c
Thanks for the KIP. Seems I am almost late to the party.
About naming (fun, fun, fun): I like the current proposal overall,
except `onBachLoaded`, but would prefer `onBatchUpdated`. It better
aligns to everything else:
- it's an update-listener, not loaded-listener
- `StateRestoreListener`
Hi Bruno,
Thanks for your observation; surely it will require a network call using
the admin client in order to know this "endOffset" and that will have an
impact on performance. We can either find a solution that has a low impact
on performance or ideally zero impact; unfortunately, I don't see a
Thanks for the KIP, Colt and Eduwer,
Are you sure there is also not a significant performance impact for
passing into the callback `currentEndOffset`?
I am asking because the comment here:
https://github.com/apache/kafka/blob/c32d2338a7e0079e539b74eb16f0095380a1ce85/streams/src/main/java/org/
Hi all,
it's a nice improvement! I don't have anything to add on top of the
previous comments, just came here to say that it seems to me consensus
has been reached and the result looks good to me.
Thanks Colt and Eduwer!
Lucas
On Sun, Oct 15, 2023 at 9:11 AM Colt McNealy wrote:
>
> Thanks, Guoz
Thanks, Guozhang. I've updated the KIP and will start a vote.
Colt McNealy
*Founder, LittleHorse.dev*
On Sat, Oct 14, 2023 at 10:27 AM Guozhang Wang
wrote:
> Thanks for the summary, that looks good to me.
>
> Guozhang
>
> On Fri, Oct 13, 2023 at 8:57 PM Colt McNealy wrote:
> >
> > Hello ther
Thanks for the summary, that looks good to me.
Guozhang
On Fri, Oct 13, 2023 at 8:57 PM Colt McNealy wrote:
>
> Hello there!
>
> Thanks everyone for the comments. There's a lot of back-and-forth going on,
> so I'll do my best to summarize what everyone's said in TLDR format:
>
> 1. Rename `onSta
Hello there!
Thanks everyone for the comments. There's a lot of back-and-forth going on,
so I'll do my best to summarize what everyone's said in TLDR format:
1. Rename `onStandbyUpdateStart()` -> `onUpdateStart()`, and do similarly
for the other methods.
2. Keep `SuspendReason.PROMOTED` and `Sus
Hello everyone,
Thanks for all your feedback for this KIP!
I think that the key to choosing proper names for this API is understanding
the terms used inside the StoreChangelogReader. Currently, this class has
two possible states: ACTIVE_RESTORING and STANDBY_UPDATING. In my opinion,
using Standby
Hello Colt,
Thanks for writing the KIP! I have read through the updated KIP and
overall it looks great. I only have minor naming comments (well,
aren't naming the least boring stuff to discuss and that takes the
most of the time for KIPs :P):
1. I tend to agree with Sophie regarding whether or no
Sohpie—
Thank you very much for such a detailed review of the KIP. It might
actually be longer than the original KIP in the first place!
1. Ack'ed and fixed.
2. Correct, this is a confusing passage and requires context:
One thing on our list of TODO's regarding reliability is to determine how
t
Hey Colt! Thanks for the KIP -- this will be a great addition to Streams, I
can't believe we've gone so long without this.
Overall the proposal makes sense, but I had a handful of fairly minor
questions and suggestions/requests
1. Seems like the last sentence in the 2nd paragraph of the Motivatio
22 matches
Mail list logo
