Yes, because event-time only advances if something makes it advance. Basically.
> On 4. Jan 2018, at 11:34, shashank agarwal wrote:
>
> But this will be wrong in my case. So I have to wait for the results until I
> receive next event.
>
>
>
>
>
> On Thu, Jan 4, 2018 at 3:53 PM, Aljoscha K
But this will be wrong in my case. So I have to wait for the results until
I receive next event.
On Thu, Jan 4, 2018 at 3:53 PM, Aljoscha Krettek
wrote:
> Think this is actually working as intended, from your earlier description
> of when results are produced: When you see Event 1.B, the wa
Think this is actually working as intended, from your earlier description of
when results are produced: When you see Event 1.B, the watermark is not
sufficiently advanced to trigger computation, only when you see Event 2.A does
the watermark advance and you get a result. This is what I would exp
@Dawid, I was using 1.3.2, I have checked on 1.4.0 also still facing the
same issue.
@Aljoscha, I have to cover the case where B can come after A from Kafka.
How I can achieve this as Event Time is not working. How should I implement
this?
A followedBy B.
As I am using kafka source and my even
Hi shashank,
What version of flink are you using? Is it possible that you are hitting this
issue: https://issues.apache.org/jira/browse/FLINK-7563 ?
Watermark semantics in CEP was buggy and events were processed only if its
timestamp was lower than current watermark while it should be lower or
What are the actual timestamps? If your BoundedOutOfOrderness extractor is
lagging by 10 seconds then only seeing Event 1.B would not trigger execution.
Only the later Event 2.A is sufficiently far ahead to trigger execution, which
you actually get.
> On 3. Jan 2018, at 17:05, shashank agarwal
Low Watermark is showing the same value which I am passing in event
"1514994744412" for all the tasks related to that stream, (No watermark) is
showing for Kafka source in UI.
So the pattern is following for CEP A followedBy B :
Event 1
- I passed A with origTimestamp X. (Low watermark updated to
Can you please check what the input watermark of your operations is? There is a
metric called "currentLowWatermark" for this.
Best,
Aljoscha
> On 3. Jan 2018, at 15:54, shashank agarwal wrote:
>
> Actually, In Kafka there are other topics also (around 5-6 topics) I am
> consuming particular t
Actually, In Kafka there are other topics also (around 5-6 topics) I am
consuming particular topic 'x' which only contains events. Other topics
have different data.
I am using two consumers in my program for 2 different topics. in first
topic x i am extracting the timestamp from origintimestamp v
Ok, but will there be events in all Kafka partitions/topics?
> On 3. Jan 2018, at 15:33, shashank agarwal wrote:
>
> Hi,
>
> Yes, Events will always carry a variable OriginTimestamp which I am using in
> the extractor. I have used fallback also in case of data missing will put
> System curren
Hi,
Yes, Events will always carry a variable OriginTimestamp which I am using
in the extractor. I have used fallback also in case of data missing will
put System current millis.
Still, it's not printing results.
Best,
Shashank
On Wed, Jan 3, 2018 at 7:40 PM, Aljoscha Krettek
wrote:
> H
Hi,
Are all the partitions always carrying data that has advancing timestamps? When
using Event-time the Kafka source (and Flink in general) needs to have steady
progress in all partitions, otherwise the watermark does not advance, which in
turn means that processing will be stalled downstream.
Hello,
I have some patterns in my program. For an example,
A followedBy B.
As I am using kafka source and my event API's using load balancers so
sometimes B comes before A. So my CEP doesn't generate any result for those
events.
I have then tried event time and applied
"BoundedOutOfOrdernessTi
13 matches
Mail list logo
