Re: Issue with CEP library

2017-09-30 Thread Kostas Kloudas
Hi Ajay, I will definitely have a look to see what is happening. And I will keep you posted. Thanks for investigating it. Kostas > On Oct 1, 2017, at 12:50 AM, Ajay Krishna wrote: > > Hi Kostas, > > Here is an example of a simple event I am trying to detect. The first and > last line are th

Re: Issue with CEP library

2017-09-30 Thread Ajay Krishna
Hi Kostas, Here is an example of a simple event I am trying to detect. The first and last line are the interesting points/events. The CEP library is not able to detect something like that. 4> (96,Sat Sep 30 22:30:25 UTC 2017,complex event,Low,32.781082,-117.01864,12.0,20.0) 4> (96,Sat Sep 30 22:

Re: how many 'run -c' commands to start?

2017-09-30 Thread Chesnay Schepler
The order in which you pass program arguments is quite important. Can you try again putting --detached directly after run? I.e. flink run --detached -c ... The reason being that arguments after the jar are treated as arguments for the user-program, and not as arguments for the job execution.

Re: Job Manager minimum memory hard coded to 768

2017-09-30 Thread Haohui Mai
We have observed the same issue in our production cluster. Filed FLINK-7743 for the fix. ~Haohui On Fri, Sep 29, 2017 at 1:18 AM Till Rohrmann wrote: > Hi Dan, > > I think Aljoscha is right and the 768 MB minimum JM memory is more of a > legacy artifact which was never properly refactored. If I

Re: Finding things not seen in the last window

2017-09-30 Thread Haohui Mai
Hi, Assuming FLINK-6465 lands, will something like SELECT COUNT(*) FROM (SELECT FIRST_VALUE(names) FROM stream) GROUP BY HOP(proctime, INTERVAL '1' MINUTE, INTERVAL '1' MINUTE) works? ~Haohui On Fri, Sep 29, 2017 at 6:52 PM Ron Crocker wrote: > Hi - > > I have a colleague who is trying to wr

Re: ArrayIndexOutOfBoundExceptions while processing valve output watermark and while applying ReduceFunction in reducing state

2017-09-30 Thread Federico D'Ambrosio
Thank you very much, Gordon. I'll try to run the job without the asynchronous snapshots first thing. As for the Event data type: it's a case class with 2 fields: a String ID and a composite case class (let's call it RealEvent) containing 3 fields of the following types: Information, which is a ca

Re: ArrayIndexOutOfBoundExceptions while processing valve output watermark and while applying ReduceFunction in reducing state

2017-09-30 Thread Tzu-Li (Gordon) Tai
Hi, Thanks for the extra info, it was helpful (I’m not sure why your first logs didn’t have the full trace, though). I spent some time digging through the error trace, and currently have some observations I would like to go through first: 1. So it seems like the ArrayIndexOutOfBoundsException

Re: ArrayIndexOutOfBoundExceptions while processing valve output watermark and while applying ReduceFunction in reducing state

2017-09-30 Thread Federico D'Ambrosio
Hi Gordon, I remembered that I had already seen this kind of exception once during the testing of the current job and fortunately I had the complete stacktrace still saved on my pc: Caused by: java.lang.ArrayIndexOutOfBoundsException: -1 at com.esotericsoftware.kryo.util.IntArray.pop(IntA