Re: Get access to unmatching events in Apache Flink Cep

.).oneOrMore() >>> .next("c").where(...) >>> .next("d").where(...) >>> .next("e").where(...); >>> >>> I have events with wrong order stream on input: >>> >>> a b d

Re: Get access to unmatching events in Apache Flink Cep

have access to events, that >> not matching. >> >> Can I have access to middle NFA state in CEP pattern, or get some other >> way to view unmatching events? >> >> Example project with CEP pattern on github >> <https://github.com/A-Kinski/apache-flink-cep/tree/main>, and my question >> on SO >> <https://stackoverflow.com/questions/78483004/get-access-to-unmatching-events-in-apache-flink-cep> >> >> Thanks in advance >> > -- С уважением, Антон.

Re: Get access to unmatching events in Apache Flink Cep

der stream on input: > > a b d c e > > On output I haven`t any matching. But I want have access to events, that > not matching. > > Can I have access to middle NFA state in CEP pattern, or get some other > way to view unmatching events? > > Example project with CEP patte

Get access to unmatching events in Apache Flink Cep

some other way to view unmatching events? Example project with CEP pattern on github <https://github.com/A-Kinski/apache-flink-cep/tree/main>, and my question on SO <https://stackoverflow.com/questions/78483004/get-access-to-unmatching-events-in-apache-flink-cep> Thanks in advance

State Memory Leak Issue in Flink CEP v1.17: Discussion & Proposed Solution

Hi, We have encountered a state memory leak issue while working with the Flink CEP (Complex Event Processing) in version 1.17. Based on code, it seems like the issue is persistent across all versions of CEP library. I've already raised a JIRA issue detailing the problem we've face

Flink CEP Operator Busy

Hello， I have defined a pattern (a->b->c->d->e->f->g) that should match within 10 minutes. However, when I check the task in the Flink WebUI, I notice that the CEP operator quickly reaches a busy state with 100% utilization. Increasing the concurrency of CEP operator does not solve the

Query Regarding Optimisation of Timer Management in Flink CEP (Version 1.17)

Hi, I am currently working with Flink CEP version 1.17, and I am in the process of load testing for potential memory leaks related to checkpoint data. While analyzing the CepOperator code, I have come across a particular pattern regarding timer registration and event processing that I believe

Re: Flink CEP Resource Utilisation Optimisation

> *发送时间:* Monday, March 27, 2023 12:38:59 AM >> *收件人:* Geng Biao >> *抄送:* user@flink.apache.org >> *主题:* Re: Flink CEP Resource Utilisation Optimisation >> >> Thanks, Geng for the quick and actionable response. >> >> I will definitely try this with Flink v

Re: Flink CEP Resource Utilisation Optimisation

- > *发件人:* Abhishek Singla > *发送时间:* Monday, March 27, 2023 12:38:59 AM > *收件人:* Geng Biao > *抄送:* user@flink.apache.org > *主题:* Re: Flink CEP Resource Utilisation Optimisation > > Thanks, Geng for the quick and actionable response. > > I wil

Re: Flink CEP Resource Utilisation Optimisation

. Best, Biao 获取 Outlook for iOS<https://aka.ms/o0ukef> 发件人: Abhishek Singla 发送时间: Monday, March 27, 2023 12:38:59 AM 收件人: Geng Biao 抄送: user@flink.apache.org 主题: Re: Flink CEP Resource Utilisation Optimisation Thanks, Geng for the quick and actionable respon

Re: Flink CEP Resource Utilisation Optimisation

an I was expecting it to be only a few MBs. Is there something I am missing here? Regards, Abhishek Singla On Sun, Mar 26, 2023 at 9:56 PM Geng Biao wrote: > Hi Abhishek, > > > > Thanks for sharing the experiment! As for the performance question, I > believe you could give a try

Re: Flink CEP Resource Utilisation Optimisation

Hi Abhishek, Thanks for sharing the experiment! As for the performance question, I believe you could give a try on Flink CEP with version >= 1.16.0, which includes the optimization introduced in FLINK-23890<https://issues.apache.org/jira/browse/FLINK-23890>. This optimization will re

Re: Flink CEP Resource Utilisation Optimisation

退订 发自我的iPhone -- Original -- From: Abhishek Singla

Flink CEP Resource Utilisation Optimisation

Hi Team, *Flink Version:* 1.15.0 *Java Version:* 1.8 *Standalone Cluster* *Task Manager:* AWS EC2 of Instance Type c5n.4xlarge (vCPU 16, Memory 42 Gb, 8 slots per TM) *CEP Scenario:* Kafka Event A followed by Kafka Event B within 10 mins *Throughput:* 20k events per second for Event A, 0 for Kafka

Re: Flink CEP Incremental Checkpoint Issue

Hi Yun Tang, Thank you for the response and yes went through some articles which explained rocksdb incremental checkpointing mechanism and makes sense w.r.t. metrics that i am seeing. Regards, Puneet > On 22-Oct-2022, at 1:26 PM, Yun Tang wrote: > > compaction

Re: Flink CEP Incremental Checkpoint Issue

hould care about the checkpoint size too much. Instead, we should care more about the output results. Best Yun Tang From: Martijn Visser Sent: Wednesday, October 19, 2022 22:03 To: Puneet Duggal Cc: user Subject: Re: Flink CEP Incremental Checkpoint Issue Hi,

Re: Flink CEP Incremental Checkpoint Issue

> > > Hi, > > > > I am working on a use case which uses Flink CEP for pattern detection. > > > > Flink Version - 1.12.1 > > Deployment Mode - Session Mode (Highly Available) > > State Backend - RocksDB > > Checkpoint Interval - 2 mins > > Ch

Re: Flink CEP Incremental Checkpoint Issue

Apologies for the mistake of calculation 120*6*2KB = 1440KB = 1.4MB > On 18-Oct-2022, at 1:35 AM, Puneet Duggal wrote: > > Hi, > > I am working on a use case which uses Flink CEP for pattern detection. > > Flink Version - 1.12.1 > Deployment Mode - Session Mode (H

Re: Flink CEP in PyFlink

/sql/queries/match_recognize/ > 2021年9月7日 下午4:44，Kamil ty 写道： > > Hello all, > > I would like to use Flink CEP for my development requirements. Is Flink CEP > supported in PyFlink? If not, are there any available workarounds? > > Kind regards > Kamil

Flink CEP in PyFlink

Hello all, I would like to use Flink CEP for my development requirements. Is Flink CEP supported in PyFlink? If not, are there any available workarounds? Kind regards Kamil

Re: Flink SQL as DSL for flink CEP

Thanks a lot Jing .  On Wednesday, June 16, 2021, 05:12:01 PM GMT+5:30, JING ZHANG wrote: Hi Dipanjan,I'm not an expert on Flink CEP, however I would like to share my immature ideas, hope it helps.Flink provides CEP library[1] which is a programmatic library which allows

Re: Flink cep checkpoint size

Hi, Dawid. Thanks for replying, happy to know you are working on this. On 2021/07/08 12:14:21, Dawid Wysakowicz wrote: > Hi, > > Sorry for the late reply. > > Indeed I found a couple of problems with clearing the state for short > lived keys. I created a JIRA[1] issue to track it and opened a

Re: Flink cep checkpoint size

Hi, Sorry for the late reply. Indeed I found a couple of problems with clearing the state for short lived keys. I created a JIRA[1] issue to track it and opened a PR (which needs test coverage before it can be merged) with fixes for those. Best, Dawid [1] https://issues.apache.org/jira/browse/

Re: Flink CEP checkpoint size

tps://issues.apache.org/jira/browse/FLINK-19970. Looks like the bug > > in scenario 1 got fixed by scenario 2 did not. There's more detail in > > this thread, > > http://deprecated-apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-cep-checkpoint-size-td441

Re: Flink CEP checkpoint size

more detail in > this thread, > http://deprecated-apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-cep-checkpoint-size-td44141.html#a44168 > . Hope that helps > > On Tue, Jul 6, 2021 at 12:44 AM Li Jim wrote: > > > > I am using Flink CEP to

Re: Flink CEP checkpoint size

-user-mailing-list-archive.2336050.n4.nabble.com/Flink-cep-checkpoint-size-td44141.html#a44168 . Hope that helps On Tue, Jul 6, 2021 at 12:44 AM Li Jim wrote: > > I am using Flink CEP to do some performance tests. > > Flink version 1.13.1. > > below is the sql: > > INSERT I

Flink CEP checkpoint size

I am using Flink CEP to do some performance tests. Flink version 1.13.1. below is the sql: INSERT INTO to_kafka SELECT bizName, wdName, wdValue , zbValue , flowId FROM kafka_source MATCH_RECOGNIZE ( PARTITION BY flow_id ORDER BY proctime MEASURES A.biz_name as bizName, A.wd_name as

Re: Flink cep checkpoint size

Hi, Mohit, Have you figured out any solusions on this problem ? I am now facing the exactly same problem , I was using Flink of version 1.12.0 and I also upgrated it to 1.13.1 but the checkpoint size is still growing. On 2021/06/02 15:45:59, "Singh, Mohit" wrote: > Hi, > > I am facing an i

Re: Flink SQL as DSL for flink CEP

Hi Dipanjan, I'm not an expert on Flink CEP, however I would like to share my immature ideas, hope it helps. Flink provides CEP library[1] which is a programmatic library which allows user define patterns based on pattern API. Besides, Flink allows user define patterns by MATCH_RECOGNIZE clau

Re: Flink SQL as DSL for flink CEP

Hi Dipanjan, Using Flink SQL's MATCH_RECOGNIZE operator is certainly a good idea if you are looking for a non-programmatic way to do CEP with Flink. On Wed, Jun 16, 2021 at 6:44 AM Dipanjan Mazumder wrote: > Hi, > > Can we say that Flink SQL is kind of a DSL overlay on flink

Flink SQL as DSL for flink CEP

Hi,     Can we say that Flink SQL is kind of a DSL overlay on flink CEP , i mean i need a DSL for flink CEP , so that i can decouple the CEP rules from code and pass them dynamically to be applied on different data streams. Flink CEP doen't have any DSL implementation , so is it that Flin

Re: DSL for Flink CEP

there any plan to grow the flink  CEP and build a friendly DSL around flink CEP by any chance. Regards Dipanjan On Wednesday, June 2, 2021, 03:22:46 PM GMT+5:30, Fabian Paul wrote: Hi Dipanjan, Unfortunately, I have no experience with Siddhi but I am not aware of any official j

Re: DSL for Flink CEP

addition, you can  > start a discussion in the community about this feature. > > Best, > Fabian >> On 2. Jun 2021, at 12:10, Dipanjan Mazumder > <mailto:java...@yahoo.com>> wrote: >> >> Hi Fabian, >> >>      Understood but is there any plan to

Re: DSL for Flink CEP

ere any plan to grow the flink CEP and build a > friendly DSL around flink CEP by any chance. > > Regards > Dipanjan > > On Wednesday, June 2, 2021, 03:22:46 PM GMT+5:30, Fabian Paul > wrote: > > > Hi Dipanjan, > > Unfortunately, I have no experience with

Re: DSL for Flink CEP

Hi Fabian,      Understood but is there any plan to grow the flink  CEP and build a friendly DSL around flink CEP by any chance. RegardsDipanjan On Wednesday, June 2, 2021, 03:22:46 PM GMT+5:30, Fabian Paul wrote: Hi Dipanjan, Unfortunately, I have no experience with Siddhi but I am

Re: DSL for Flink CEP

-siddhi library > has limited support for flnk versions and i will either need to fix the > library or get tied to a fix version of Flink to use th library. > > I am looking at Flink CEP as an option , and also came across a Flink CEP > DSL library (https://github.com/phil3k3/flin

DSL for Flink CEP

Hi ,    I am currently using Siddhi CEP with flink , but the flink-siddhi library has limited support for flnk versions and i will either need to fix the library or get tied to a fix version of Flink to use th library.  I am looking at Flink CEP as an option , and also came across a Flink CEP

flink cep and out of order events

does the default within behavior of flink cep handle out of order events (relative to event time)? obviously, it'd be best if the event time was guaranteed correct, but sometimes it's too difficult to do. do people end up writing different patterns with some event orderings reversed

Re[2]: Flink CEP: can't process PatternStream (v 1.12, EventTime mode)

аевич wrote: >>Hello, >>  >>I’ve already asked the question today and got the solve:  >>http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-CEP-can-t-process-PatternStream-td41722.html >> , and it’s clean for me how PatternStream works wit

Re: Flink CEP: can't process PatternStream

Hello, David. Yes, I’m using 1.12. And my code is now working. Thank you very much for your comment. Yuri L. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Flink CEP: can't process PatternStream (v 1.12, EventTime mode)

/2021 13:21, Люльченко Юрий Николаевич wrote: > Hello, >   > I’ve already asked the question today and got the > solve:  > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-CEP-can-t-process-PatternStream-td41722.html > <http://apache-flink-user-maili

Flink CEP: can't process PatternStream (v 1.12, EventTime mode)

Hello,   I’ve already asked the question today and got the solve:  http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-CEP-can-t-process-PatternStream-td41722.html , and it’s clean for me how PatternStream works with ProcessTime.   But I need help again, I can’t write

Re: Flink CEP: can't process PatternStream

/02/2021 09:18, Люльченко Юрий Николаевич wrote: > Hello everyone. >   > I’m trying to use Flink Cep library and I want to fetch some events by > pattern. At first I’ve created a simple HelloWorld project. But I have > a problem exactly like it described > here:  > https://stacko

Flink CEP: can't process PatternStream

Hello everyone.   I’m trying to use Flink Cep library and I want to fetch some events by pattern. At first I’ve created a simple HelloWorld project. But I have a problem exactly like it described here:  https://stackoverflow.com/questions/39575991/flink-cep-no-results-printed   You can see my

Re: [Stateful Functions] Using Flink CEP

41 AM Igal Shilman wrote: > Hi, > > I'm not familiar with the other library that you have mentioned, and > indeed using Flink CEP from within a stateful function is not possible > within a single Flink job, as Gordon mentioned. > > I'm wondering what aspects of CEP

Re: [Stateful Functions] Using Flink CEP

11:07 PM Tzu-Li (Gordon) Tai wrote: > Hi! > > It isn't possible to use Flink CEP within Stateful Functions. > > That could be an interesting primitive, to add CEP-based function > constructs. > Could your briefly describe what you are trying to achieve? > > On the ot

Re: [Stateful Functions] Using Flink CEP

Hi, I'm not familiar with the other library that you have mentioned, and indeed using Flink CEP from within a stateful function is not possible within a single Flink job, as Gordon mentioned. I'm wondering what aspects of CEP are you interested in? Because essentially a stateful funct

Re: [Stateful Functions] Using Flink CEP

Hi! It isn't possible to use Flink CEP within Stateful Functions. That could be an interesting primitive, to add CEP-based function constructs. Could your briefly describe what you are trying to achieve? On the other hand, there are plans to integrate Stateful Functions more closely wit

Re: [Stateful Functions] Using Flink CEP

I am leaning towards using Siddhi as a library, but I would really love to stick with Flink CEP, or at least the specific CEP mechanism that Flink CEP uses. Exploring the codebase of Flink CEP wasn't much promising on this end. -- [image: MotaWord] Oytun Tez M O T A W O R D | CTO & C

[Stateful Functions] Using Flink CEP

Hi there, I was wondering if I could somehow use CEP within a Function. Have you experimented with this before? Or, do you have any suggestions to do CEP within a Function? I am looking for a standalone library now. Oytun -- [image: MotaWord] Oytun Tez M O T A W O R D | CTO & Co-Founder oy.

Re: can we do Flink CEP on event stream or batch or both?

same processing semantics. > > Best, > Fabian > > > Am Di., 30. Apr. 2019 um 06:49 Uhr schrieb kant kodali >: > >> Hi All, >> >> I have the following questions. >> >> 1) can we do Flink CEP on event stream or batch? >> 2) If we can do stream

Re: can we do Flink CEP on event stream or batch or both?

. Best, Fabian Am Di., 30. Apr. 2019 um 06:49 Uhr schrieb kant kodali : > Hi All, > > I have the following questions. > > 1) can we do Flink CEP on event stream or batch? > 2) If we can do streaming I wonder how long can we keep the stream > stateful? I also wonder if anyon

can we do Flink CEP on event stream or batch or both?

Hi All, I have the following questions. 1) can we do Flink CEP on event stream or batch? 2) If we can do streaming I wonder how long can we keep the stream stateful? I also wonder if anyone successfully had done any stateful streaming for days or months(with or without CEP)? or is stateful

Re: Apache Flink - CEP vs SQL detecting patterns

Hi, Unfortunately those are just ignored. The timed out partial matches are not emitted. Best, Dawid On 20/04/2019 19:49, M Singh wrote: > Dawid: > > So, what happens when there is a timeout - is there any value/field in > the resulting data stream that indicates that this was a timeout ? > > T

Re: Apache Flink - CEP vs SQL detecting patterns

Dawid: So, what happens when there is a timeout - is there any value/field in the resulting data stream that indicates that this was a timeout ? Thanks On Tuesday, April 16, 2019, 10:12:58 AM EDT, Dawid Wysakowicz wrote: Hi Mans, Yes you are right. That feature is not available i

Re: Apache Flink - CEP vs SQL detecting patterns

Hi Mans, Yes you are right. That feature is not available in SQL, as there is no such feature in SQL standard. The only addition to SQL standard we introduce so far is the WITHIN clause. We might introduce the timed out patterns some time in the future, but personally I am not aware of such plans.

Apache Flink - CEP vs SQL detecting patterns

Hi: I am looking at the documentation of the CEP and there is way to access patterns which have timeout.  But I could  not find similar capability in the Table and SQL interface detecting patterns.  I am assuming that the CEP interface is more comprehensive and complete than the SQL/Table interf

Re: Flink CEP pattern design question

I think what you ask for is something like timing out greedy[1] quantifier, which is not supported. As a rather dirty workaround you could try sth like: Pattern .begin[Event]("start") .where(_._.getName == "a") .oneOrMore.opt

Flink CEP pattern design question

Currently I'm designing a CEP pattern to satisfy our business needs. Basically, there's two events let's call it a and b. Both a and b can have zero or multiple entries in the log. For input {a,b1,b2}, I want to get the output of {a,b1,b2} For input {b1,b2}, I want to get the output of {b1,b2} afte

Re: [flink-cep] - What is the difference between GroupPattern and Pattern?

_ljy 写道： > > Hi, all > I notice that we have GroupPattern, which is subclass of Pattern in > flink-cep module. I’m not very sure about the meaning of GroupPattern and its > necessity. I will appreciate if anyone can list some examples for me to help > understand this? >

[flink-cep] - What is the difference between GroupPattern and Pattern?

Hi, all I notice that we have GroupPattern, which is subclass of Pattern in flink-cep module. I’m not very sure about the meaning of GroupPattern and its necessity. I will appreciate if anyone can list some examples for me to help understand this? Thanks! Best Regards, Jiayi Liao

Re: Dynamic Rules Creation For Flink CEP

I have written a blogpost though its not used CEP but I hope you can apply same thing for CEP as well. http://dhanuka84.blogspot.com/2019/02/rule-execution-as-streaming-process.html?m=1 On Tue, 12 Feb 2019, 11:46 Dian Fu This feature has not been supported yet. There is a JIRA( > https://issu

Re: Dynamic Rules Creation For Flink CEP

This feature has not been supported yet. There is a JIRA(https://issues.apache.org/jira/browse/FLINK-7129) for this feature. > 在 2019年2月12日，上午9:32，Titus Rakkesh 写道： > > Dears, > We are having the use case of creating and uploading new patterns > dynamically and change the app behavior in

Dynamic Rules Creation For Flink CEP

Dears, We are having the use case of creating and uploading new patterns dynamically and change the app behavior in runtime. Is the current Flink version support that? Thanks

Re: Flink CEP : Doesn't generate output

Can you provide us a self-contained reproducing example? (preferably as elementary as possible) On 22.01.2019 18:58, dhanuka ranasinghe wrote: Hi All, I have used Flink CEP to filter some events and generate some alerts based on certain conditions. But unfortunately doesn't print any r

Re: Flink CEP : Doesn't generate output

Thank you for the clarification. On Thu, 24 Jan 2019, 12:44 Dian Fu Hi Dhanuka, > > From the code you shared, it seems that you're using event time. The > processing of elements is triggered by watermark in event time and so you > should define how to generate the watermark, i.e with > DataStream

Re: Flink CEP : Doesn't generate output

Hi Dhanuka, From the code you shared, it seems that you're using event time. The processing of elements is triggered by watermark in event time and so you should define how to generate the watermark, i.e with DataStream.assignTimestampsAndWatermarks Regards, Dian > 在 2019年1月23日，上午1:58，dhanuka

Flink CEP : Doesn't generate output

Hi All, I have used Flink CEP to filter some events and generate some alerts based on certain conditions. But unfortunately doesn't print any result. I have attached source code herewith, could you please help me on this. package org.monitoring.stream.anal

Re: Apache Flink CEP how to detect if event did not occur within x seconds?

gt; > > Regards, > Dian > >> 在 2018年12月13日，上午12:06，Andrey Zagrebin >> mailto:and...@data-artisans.com>> 写道： >> >> Hi Florin, >> >> I think Dawid might help you. I am pulling him into the discussion. >> >> Best,

Re: Apache Flink CEP how to detect if event did not occur within x seconds?

; >> Best, >> Andrey >> >>> On 12 Dec 2018, at 16:24, Spico Florin >> <mailto:spicoflo...@gmail.com>> wrote: >>> >>> Hello! >>> I'm using the same questions as in this stackoverflow >>> post  >>> https://stackov

Re: Apache Flink CEP how to detect if event did not occur within x seconds?

k(maxTimestamp); >} > } > > > Regards, > Dian > > 在 2018年12月13日，上午12:06，Andrey Zagrebin 写道： > > Hi Florin, > > I think Dawid might help you. I am pulling him into the discussion. > > Best, > Andrey > > On 12 Dec 2018, at 16:24, Spico Florin wrote

Re: Apache Flink CEP how to detect if event did not occur within x seconds?

:spicoflo...@gmail.com>> wrote: >> >> Hello! >> I'm using the same questions as in this stackoverflow post >> https://stackoverflow.com/questions/50356981/apache-flink-cep-how-to-detect-if-event-did-not-occur-within-x-seconds?rq=1 >> >> <https

Re: Apache Flink CEP how to detect if event did not occur within x seconds?

Hi Florin, I think Dawid might help you. I am pulling him into the discussion. Best, Andrey > On 12 Dec 2018, at 16:24, Spico Florin wrote: > > Hello! > I'm using the same questions as in this stackoverflow post > https://stackoverflow.com/questions/50356981/apache-flink-

Apache Flink CEP how to detect if event did not occur within x seconds?

Hello! I'm using the same questions as in this stackoverflow post https://stackoverflow.com/questions/50356981/apache-flink-cep-how-to-detect-if-event-did-not-occur-within-x-seconds?rq=1, due to fact that I need the same functionality. My use case is to detect when an device is disconnected

Re: Something wrong with the until condition FLINK-CEP

Sorry, it seems that I misunderstood the concept of the composition of the until condition and oneOrMore. Original Message Sender:bupt_ljybupt_...@163.com Recipient:useru...@flink.apache.org Date:Tuesday, Dec 11, 2018 14:00 Subject:Something wrong with the until condition FLINK-CEP Hi all, I

Something wrong with the until condition FLINK-CEP

Hi all, I seem to find a problem of until condition in testGreedyUntilZeroOrMoreWithDummyEventsAfterQuantifier in GreedyITCase.java. I modify the unit test a little bit like this: @Test public void testGreedyUntilZeroOrMoreWithDummyEventsAfterQuantifier() throws Exception { ListStreamRecord

Re: Flink CEP support pattern match involving fields of previous events

ty > like: |.where(second_evt->evt.getTemperature() == > first_evt->evt.getTemperature()"| >    > https://stackoverflow.com/questions/42065388/when-might-flink-support-pattern-match-involving-fields-of-previous-events > and  > https://stackoverflow.com/questions/38770195/apa

Flink CEP support pattern match involving fields of previous events

Temperature()" https://stackoverflow.com/questions/42065388/when-might-flink-support-pattern-match-involving-fields-of-previous-events and https://stackoverflow.com/questions/38770195/apache-flink-cep-detect-same-events If, yes which version support this feature and what is the API syntax? Thanks. Florin

Re: [flink-cep] Flick CEP support for the group By operator

Hey, As a matter of fact, you do not need a Flink's CEP library to run the same query. The same functionality can be achieved by simple tumbling window with a “median” aggregate (“median" you would have to implement by your self). https://ci.apache.org/projects/flink/flink-docs-stable/dev/strea

Re: [flink-cep] Flick CEP support for the group By operator

Hi, Yes, sure. Just use CEP on top of KeyedStream. Take a look at `keyBy`: https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/operators/ Piotrek > On 23 Nov 2018, at 16:04, Spico Florin wrote: > > Hello! > > I'm using Flink 1.4.2 and I would like to use a group by operat

[flink-cep] Flick CEP support for the group By operator

Hello! I'm using Flink 1.4.2 and I would like to use a group by operator based on value of my event stream. The functionality that I would like to achieve is similar to the following Esper EPL (excerpt http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggr

Re: Flink CEP Watermark Exception

abtu...@mail.gmail.com%3E > > > On Thu, Nov 1, 2018 at 4:44 AM Dawid Wysakowicz > wrote: > >> Hi Austin, >> >> Could you elaborate a bit more what do you mean by "after a checkpoint >> fails", what is the reason why checkpoint fails? Would it be pos

Re: Flink CEP Watermark Exception

tps://mail-archives.apache.org/mod_mbox/flink-user/201702.mbox/%3CCAGr9p8CzT3=cr+cOas=3k0becmiybfv+kt1fxv_rf8xabtu...@mail.gmail.com%3E >   > > On Thu, Nov 1, 2018 at 4:44 AM Dawid Wysakowicz > mailto:dwysakow...@apache.org>> wrote: > > Hi Austin, > > Could you elabor

Re: Flink CEP Watermark Exception

t do you mean by "after a checkpoint > fails", what is the reason why checkpoint fails? Would it be possible for > you to prepare some reproducible example for that problem? Finally, I would > also recommend trying out Flink 1.6.x, as we reworked the underlying > structure

Re: Flink CEP Watermark Exception

worked the underlying structure for CEP - SharedBuffer. Best, Dawid On 30/10/2018 20:59, Austin Cawley-Edwards wrote: > Following up, we are using Flink 1.5.0 and Flink-CEP 2.11. > > Thanks, > Austin > > On Tue, Oct 30, 2018 at 3:58 PM Austin Cawley-Edwards > mailto:austin.caw..

Re: Flink CEP Watermark Exception

Following up, we are using Flink 1.5.0 and Flink-CEP 2.11. Thanks, Austin On Tue, Oct 30, 2018 at 3:58 PM Austin Cawley-Edwards < austin.caw...@gmail.com> wrote: > Hi there, > > > We have a streaming application that uses CEP processing but are getting this > error fair

Flink CEP Watermark Exception

Hi there, We have a streaming application that uses CEP processing but are getting this error fairly frequently after a checkpoint fails, though not sure if it is related. We have implemented both `hashCode` and `equals()` using `Objects.hash(...properties)` and basic equality, respectively. Has

Flink CEP detect multiple pattern witin their specific time window

Hi, I have 2 different pattern, 1st pattern should appear within 4 hours and another pattern should appear within 24 hour. I can go ahead and create 2 jobs running on flink cluster. But is it possible to use same job to detect both patter within their own time window as my source stream and elast

Re: Flink CEP window for 1 working day

Hi, I don't think the CEP library is that flexible, but I loop in Kostas (CC) who knows more about it. I'm not exactly sure what you mean by "manipulate" event-time, but I don't think that's necessary. You can implement rules also with state and timers in the ProcessFunction. The function ingests

Re: Flink CEP window for 1 working day

Thanks Fabian, So by non working day, I mean, I have a list of non working day in a year, which I can use to compare. I am very new to Flink and Flick CEP. Initially I thought there is a way to have within(time) value expression dynamically. So now I guess that's not possible. If I understand c

Re: Flink CEP window for 1 working day

Hi, I'm afraid, Flink CEP does not distinguish work days from non-work days. Of course, you could implement the logic in a DataStream program (probably using ProcessFunction). Best, Fabian 2018-03-20 15:44 GMT+01:00 shishal : > I am using flink CEP , and to match a event pattern in gi

Flink CEP window for 1 working day

I am using flink CEP , and to match a event pattern in given time window we use *.within(Time.days(1))* Now in one of the case I need to wait for 1 working day instead of 1 day. Is there any way to do that in Flink CEP? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4

design flink cep pattern for transaction life cycle

0down votefavorite <https://stackoverflow.com/questions/49273875/design-flink-cep-pattern-for-transaction#> I have a scenario where I need to detect following pattern for a transaction. My basic transaction event look like this => *Transaction(id,status, time)* Transaction goe

Flink CEP pattern with multiple temporal constraint

inkCEP for this problem. I have gone through their doc https://ci.apache.org/projects/flink/flink-docs- release-1.4/dev/libs/cep.html But still I am not sure how to design this pattern. I hope it is possible to design pattern with Flink CEP? Also if I am not wrong , I need to use PatternTimeoutFunc

Re: Flink CEP with files and no streams?

you have > any good Scala code examples about this ? > > > > Esa > > > > *From:* Fabian Hueske [mailto:fhue...@gmail.com] > *Sent:* Wednesday, February 7, 2018 11:21 AM > *To:* Esa Heikkinen > *Cc:* user@flink.apache.org > *Subject:* Re: Flink CEP with files an

RE: Flink CEP with files and no streams?

Hi Thanks for the reply, but because I am a newbie with Flink, do you have any good Scala code examples about this ? Esa From: Fabian Hueske [mailto:fhue...@gmail.com] Sent: Wednesday, February 7, 2018 11:21 AM To: Esa Heikkinen Cc: user@flink.apache.org Subject: Re: Flink CEP with files and

Re: Flink CEP with files and no streams?

Hi Esa, you can also read files as a stream. However, you have to be careful in which order you read the files and how you generate watermarks. The easiest approach is to implement a non-parallel source function that reads the files in the right order and generates watermarks. Things become more t

Flink CEP with files and no streams?

Hello I am trying to use CEP of Flink for log files (as batch job), but not for streams (as realtime). Is that possible ? If yes, do you know examples Scala codes about that ? Or should I convert the log files (with time stamps) into streams ? But how to handle time stamps in Flink ? If I can n

Re: Flink CEP exception during RocksDB update

Hi Varun, The branch I previously sent you has been now merged to the master. So could you try the master and tell us if you see any change in the behavior? Has the problem been fixed, or has the message of the exception changed? Thanks, Kostas > On Jan 29, 2018, at 10:09 AM, Kostas Kloudas

Re: Flink CEP exception during RocksDB update

Hi again Varun, I am investigating the problem you mentioned and I found a bug in the SharedBuffer, but I am not sure if it is the only bug that affects you. Could you please try this branch https://github.com/kl0u/flink/tree/cep-inv and let me know

  1   2   3   >