Can someone confirm that each
partition has its own stream time and that the stream time for a partition
only advances when a record is written to the partition after the window
closes?
That's correct.
On 5/21/24 10:11 AM, Chad Preisler wrote:
After reviewing the logs, I think I understand
After reviewing the logs, I think I understand what happens with the
repartition topics. Looks like they will be assigned to one or more
instances. In my example I ran three instances of the application (A, B,
C). Looks like the two repartition topics got assigned to A and B. The six
partitions fro
See one small edit below...
On Tue, May 21, 2024 at 10:25 AM Chad Preisler
wrote:
> Hello,
>
> I think the issue is related to certain partitions not getting records to
> advance stream time (because of low volume). Can someone confirm that each
> partition has its own stream time and that the s
Hello,
I think the issue is related to certain partitions not getting records to
advance stream time (because of low volume). Can someone confirm that each
partition has its own stream time and that the stream time for a partition
only advances when a record is written to the topic after the windo
How do you know this?
First thing we do is write a log message in the value joiner. We don't see
the log message for the missed records.
Well, for left/right join results, the ValueJoiner would only be called
when the window is closed... And for invalid input (or late record, ie,
which arrive
Hello,
We did some testing in our test environment today. We are seeing some
records processes where only one side of the join has a record. So that's
good. However, we are still seeing some records get skipped. They never hit
the value joiner (we write a log message first thing in the value joine
Thanks for the information. I ran the code using Kafka locally. After
submitting some records inside and outside of the time window and grace,
the join performed as expected when running locally.
That gives some hope :)
However, they never get into the join.
How do you know this?
Did you
Matthias,
Thanks for the information. I ran the code using Kafka locally. After
submitting some records inside and outside of the time window and grace,
the join performed as expected when running locally.
I'm not sure why the join is not working as expected when running against
our actual broker
I expect the join to
execute after the 25 with one side of the join containing a record and the
other being null
Given that you also have a grace period of 5 minutes, the result will
only be emitted after the grace-period passed and the window is closed
(not when window end time is reached).
Hello,
I have a KStream to KStream outer join with a time difference of 25 minutes
and 5 minutes of grace. When I get a record for one side of the join, but
don't get a record on the other side of the join, I expect the join to
execute after the 25 with one side of the join containing a record an
10 matches
Mail list logo
