Re: Flink SSL Setup on a standalone cluster

Hi, The passwords are shown in plain text in logs , is this fixed in newer versions of flink (I am using 1.3.2) Also, please let me know the answer to my previous queries in this mail chain Regards, Vinay Patil On Mon, Mar 19, 2018 at 7:35 PM, Vinay Patil wrote: > Hi, > > When I set ssl.verif

"dynamic" bucketing sink

Hi all, I'm using the nice topic pattern feature on the KafkaConsumer to read from multiple topics, automatically discovering new topics added into the system. At the end of the processing I'm sinking the result into a Hadoop Filesystem using a BucketingSink. All works great until I get the requ

Re: Incremental checkpointing performance

Hi Miyuru, regarding "state.backend", I was looking at version 1.5 docs and some things changed compared to 1.3. The "Asynchronous RocksDB snapshot ..." messages only occur with full snapshots, i.e. non-incremental, and I verified this for your program as well. There are some issues with your proj

RE: Lucene SPI class loading fails with shaded flink-connector-elasticsearch

Hi Gordon, hi Till, Thanks for your feedback. I am happy to contibute by precising how the bug occured, if it might help. First, to describe a bit more what does my Flink job, there is in a part of its execution plan a ProcessFunction which basically stores the events as Lucene documents in

Flink CEP detect multiple pattern witin their specific time window

Hi, I have 2 different pattern, 1st pattern should appear within 4 hours and another pattern should appear within 24 hour. I can go ahead and create 2 jobs running on flink cluster. But is it possible to use same job to detect both patter within their own time window as my source stream and elast

Re: Lucene SPI class loading fails with shaded flink-connector-elasticsearch

Hi Manuel, Thanks a lot for reporting this! Yes, this issue is most likely related to the recent changes to shading the Elasticsearch connector dependencies, though it is a bit curious why I didn’t bump into it before while testing it. The Flink job runs Lucene queries on a data stream which e

Re: Example PopularPlacesFromKafka fails to run

Thanks for the feedback! Fabian 2018-03-23 8:02 GMT+01:00 James Yu : > Hi, > > When I proceed further to timePrediction exercise (http://training.data- > artisans.com/exercises/timePrediction.html), I realize that the > nycTaxiRides.gz's > format is fine. > The problem is in TaxiRide.toString()

Re: Restart hook and checkpoint

Yes, that would be great! Thank you, Fabian 2018-03-23 3:06 GMT+01:00 Ashish Pokharel : > Fabian, that sounds good. Should I recap some bullets in an email and > start a new thread then? > > Thanks, Ashish > > > On Mar 22, 2018, at 5:16 AM, Fabian Hueske wrote: > > Hi Ashish, > > Agreed! > I th

Re: Example PopularPlacesFromKafka fails to run

Hi, When I proceed further to timePrediction exercise ( http://training.data-artisans.com/exercises/timePrediction.html), I realize that the nycTaxiRides.gz's format is fine. The problem is in TaxiRide.toString(), the columns were serialized in wrong order. Hence the data persisted in Kafka has wr