Re: At what point do watermarks get injected into the stream?

2017-06-11 Thread Fabian Hueske
Each parallel instance of a TimestampAssigner independently assigns timestamps. After a shuffle, operators forward the minimum watermark across all input connections. For details have a look at the watermarks documentation [1]. Best, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-rele

Re: Java 8 lambdas for CEP patterns won't compile

2017-06-11 Thread Ted Yu
Looking at docs/dev/libs/cep.md , there're 3 examples using lambda. Here is one: Pattern pattern = Pattern.begin("start").where(evt -> evt.getId() == 42) Your syntax should be supported. I haven't found such example in test code, though. FYI On Sun, Jun 11, 2017 at 2:42 PM, David Koch wrote:

Java 8 lambdas for CEP patterns won't compile

2017-06-11 Thread David Koch
Hello, I cannot get patterns expressed as lambdas like: Pattern pattern1 = Pattern. begin("start") .where(evt -> evt.key.length() > 0) .next("last").where(evt -> evt.key.length() > 0).within(Time.seconds(5)); to compile with Flink 1.3.0. My IDE doesn't handle it and building on command l

Re: Listening to timed-out patterns in Flink CEP

2017-06-11 Thread David Koch
Hello, It's been a while and I have never replied on the list. In fact, the fix committed by Till does work. Thanks! On Tue, Apr 25, 2017 at 9:37 AM, Moiz Jinia wrote: > Hey David, > Did that work for you? If yes could you share an example. I have a similar > use case - need to get notified of

Re: Problem with WebUI

2017-06-11 Thread Dawid Wysakowicz
Thx, for suggestion. I've already resolved it. It was indeed, netty included with flink-connector-elasticsearch5. 11.06.2017 7:09 PM "Chesnay Schepler" napisał(a): > This looks like a dependency conflict to me. Try checking whether anything > you use depends on netty. > > On 09.06.2017 17:42, Da

Re: Problem with WebUI

2017-06-11 Thread Chesnay Schepler
This looks like a dependency conflict to me. Try checking whether anything you use depends on netty. On 09.06.2017 17:42, Dawid Wysakowicz wrote: I had a look into yarn logs and I found such exception: 2017-06-09 17:10:20,922 ERROR org.apache.flink.runtime.webmonitor.files.StaticFileSe

Re: At what point do watermarks get injected into the stream?

2017-06-11 Thread Ray Ruvinskiy
Thanks for the explanation, Fabian. Suppose I have a parallel source that does not inject watermarks, and the first operation on the DataStream is assignTimestampsAndWatermarks. Does each parallel task that makes up the source independently inject watermarks for the records that it has read? Su

Re: Fink: KafkaProducer Data Loss

2017-06-11 Thread ninad
Thanks Gordon. On Jun 11, 2017 9:11 AM, "Tzu-Li (Gordon) Tai [via Apache Flink User Mailing List archive.]" wrote: > Hi Ninad, > > Thanks for the logs! > Just to let you know, I’ll continue to investigate this early next week. > > Cheers, > Gordon > > On 8 June 2017 at 7:08:23 PM, ninad ([hidden

Re: Fink: KafkaProducer Data Loss

2017-06-11 Thread Tzu-Li (Gordon) Tai
Hi Ninad, Thanks for the logs! Just to let you know, I’ll continue to investigate this early next week. Cheers, Gordon On 8 June 2017 at 7:08:23 PM, ninad (nni...@gmail.com) wrote: I built Flink v1.3.0 with cloudera hadoop and am able to see the data loss. Here are the details: *tmOneClou

Re: In-transit Data Encryption in EMR

2017-06-11 Thread Tzu-Li (Gordon) Tai
Hi Vinay, Apologies for the inactivity on this thread, I was occupied with some critical fixes for 1.3.1. 1. Can anyone please explain me how do you test if SSL is working correctly ? Currently I am just relying on the logs. AFAIK, if any of the SSL configuration settings are enabled (*.ssl.en