Re: Flink Iterations vs. While loop

2016-09-09 Thread Drewes, Dan Benedikt
Hi Till, you're right, my implementation wouldn't scale well for a very large number of features.Thank you for that hint! However, i'm not using that much features, so this shouldn't be the cause for the strange behaviour. Yes, the 30 Minutes is the time for all jobs together. It's the time di

Re: Firing windows multiple times

2016-09-09 Thread Vishnu Viswanath
Hi, Sorry, I was little busy lately. I have most part of FLIP-4 ready except ProcessingTimeEvictor case. I was trying to find the best way to set the ProcessingTime as per the suggestion from Aljoscha then I got busy with some other work. We can definitely work together on that. Thanks Vishnu O

Why tuples are not ignored after watermark?

2016-09-09 Thread Saiph Kappa
Hi, I have a streaming (event time) application where I am receiving events with the same assigned timestamp. I receive 1 events in total on a window of 5 minutes, but I emit water mark when 9000 elements have been received. This watermark is 6 minutes after the assigned timestamps. My questio

Simple batch job hangs if run twice

2016-09-09 Thread Yassine MARZOUGUI
Hi all, When I run the following batch job inside the IDE for the first time, it outputs results and switches to FINISHED, but when I run it again it is stuck in the state RUNNING. The csv file size is 160 MB. What could be the reason for this behaviour? public class BatchJob { public static

Re: Administration of running jobs

2016-09-09 Thread Marek Svitok
Hi Robert. Thank you very much for your reply. I'll take a look at it. Marek. On 9.9.2016 12:09, Robert Metzger wrote: Hi Marek, You can use the RemoteExecutionEnvironment to submit a job programatically to a Flink cluster. there is also some ongoing work to programatically control submitt

Re: Distributed Cache support for StreamExecutionEnvironment

2016-09-09 Thread Robert Metzger
Hi Swapnil, there's no support for something like DistributedCache in the DataStream API. However, as a workaround, you can rely on the RichFunction's open() method's to load such data directly from a distributed file system. Regards, Robert On Fri, Sep 9, 2016 at 8:13 AM, Swapnil Chougule wrot

Re: Administration of running jobs

2016-09-09 Thread Robert Metzger
Hi Marek, You can use the RemoteExecutionEnvironment to submit a job programatically to a Flink cluster. there is also some ongoing work to programatically control submitted jobs: https://issues.apache.org/jira/browse/FLINK-4272. But for now you would probably need to hack something using the Job

Re: Unable to find KvState flink Queryablestate

2016-09-09 Thread Robert Metzger
Hi, I fear that you have to look into the Flink code to understand what's going on. Queryable state is an experimental, undocumented feature, and the committer who mainly implemented it (Ufuk) is currently on vacation. There's probably some problem with the registration of the state at the jobman

Re: NoClassDefFoundError with ElasticsearchSink on Yarn

2016-09-09 Thread Fabian Hueske
+1 I ran into that issue as well. Would be great to have that in the docs! 2016-09-09 11:49 GMT+02:00 Robert Metzger : > Hi Steffen, > > I think it would be good to add it to the documentation. > Would you like to open a pull request? > > > Regards, > Robert > > > On Mon, Sep 5, 2016 at 10:26 PM,

Re: NoClassDefFoundError with ElasticsearchSink on Yarn

2016-09-09 Thread Robert Metzger
Hi Steffen, I think it would be good to add it to the documentation. Would you like to open a pull request? Regards, Robert On Mon, Sep 5, 2016 at 10:26 PM, Steffen Hausmann < stef...@hausmann-family.de> wrote: > Thanks Aris for your explanation! > > A guava version mismatch was indeed the pr

Re: Exception in CEP 1.1.2

2016-09-09 Thread Till Rohrmann
Great to hear :-) I was already afraid that I've overlooked another window boundary condition which I got wrong. If you should encounter other problems, then let me know. Cheers, Till On Fri, Sep 9, 2016 at 9:59 AM, Gyula Fóra wrote: > Hi, > So we cant seem to be able to reproduce the error aft

Re: Firing windows multiple times

2016-09-09 Thread Aljoscha Krettek
Hi, I'd be very happy to give you pointers for FLIP-2 and FLIP-4. Why don't you start a separate thread on the dev list so that we don't hijack this thread. For FLIP-4 we also have to coordinate with Vishnu, he was driving FLIP-4 but lately everyone has been a bit inactive on that. Let's see if he

Re: Exception in CEP 1.1.2

2016-09-09 Thread Gyula Fóra
Hi, So we cant seem to be able to reproduce the error after clearing the local maven cache. It works now :) Gyula Gyula Fóra ezt írta (időpont: 2016. szept. 7., Sze, 22:05): > Interestingly on my local machine I could not reproduce the problem, maybe > it was some build issue on the other machi