atermark strategy
> - and replaces it with something that is arbitrary (at this point
> it is hard to guess the correct max lateness that is a mixture of the
> events from multiple Kafka partitions)
>
>
>
> Concusion:
>
> The only way to make the event tim
manner.
Hope this helps
Thias
From: HG
Sent: Tuesday, March 29, 2022 1:07 PM
To: Schwalbe Matthias
Cc: user
Subject: Re: Watermarks event time vs processing time
⚠EXTERNAL MESSAGE – CAUTION: Think Before You Click ⚠
Hello Matthias,
When I remove all the watermark strategies it does not
and it tells all downstream operators
>- Idleness specification is only useful if a respective operator has
>another source of valid watermarks (i.e. after a union of two streams, one
>active/one idle ….). this is not your case
>
>
>
> I hope this clarifies you
n’t
make progress from this steam, and it tells all downstream operators
* Idleness specification is only useful if a respective operator has
another source of valid watermarks (i.e. after a union of two streams, one
active/one idle ….). this is not your case
I hope this clarifies your situat
- Coming back to your idleness configuration: it only means that the
>incoming stream becomes idle == timeless after a while … i.e. watermarks
> won’t make progress from this steam, and it tells all downstream operators
>- Idleness specification is only useful if a respective
two streams, one
active/one idle ….). this is not your case
I hope this clarifies your situation.
Cheers
Thias
From: HG
Sent: Mittwoch, 16. März 2022 10:06
To: user
Subject: Watermarks event time vs processing time
⚠EXTERNAL MESSAGE – CAUTION: Think Before You Click ⚠
Hi,
I read from a
Hi,
I read from a Kafka topic events that are in JSON format
These event contain a handling time (aka event time) in epoch milliseconds,
a transaction_id and a large nested JSON structure.
I need to group the events by transaction_id, order them by handling time
and calculate the differences in ha
