Re: FlinkKafkaProducer usage

2017-02-01 Thread Dawid Wysakowicz
Have a look at https://github.com/apache/beam/blob/master/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java . 02.02.2017 1:07 AM "Boris Lublinsky" napisał(a): > I am trying to write a quick sample of streaming word count using Beam > APIs and FlinkBeamRunner. > The probl

Re: Support for Auto scaling

2017-02-01 Thread Tzu-Li (Gordon) Tai
Hi Sandeep! While auto scaling jobs in Flink still isn’t possible, in Flink 1.2 you will be able to rescale jobs by stopping and restarting. This works by taking a savepoint of the job before stopping the job, and then redeploy the job with a higher / lower parallelism using the savepoint. Upon

Chicago Hands on Apache Flink Workshop

2017-02-01 Thread Trevor Grant
Any one who is going to be in or around Chicago 2/21: Joe Olson is putting on a workshop for our local Flink meeup- drop by if you can! https://www.meetup.com/Chicago-Apache-Flink-Meetup-CHAF/events/237385428/ Trevor Grant Data Scientist https://github.com/rawkintrevo http://stackexchange.com/us

Support for Auto scaling

2017-02-01 Thread Meghashyam Sandeep V
Hi Guys, I currently run flink 1.1.4 streaming jobs in EMR in AWS with yarn. I understand that EMR supports auto scaling but Flink doesn't. Is there a plan for this support in 1.2. Thanks, Sandeep

FlinkKafkaProducer usage

2017-02-01 Thread Boris Lublinsky
I am trying to write a quick sample of streaming word count using Beam APIs and FlinkBeamRunner. The problem that I am getting into is that apply("Write to Kafka", Write.to(UnboundedFlinkSink.of(kafkaSink))) Does not work in this way - it assumes bounded stream and mine is unbounded. I have not

Re: Graphite reporter recover from broken pipe

2017-02-01 Thread Maciek Próchniak
Starting with flink 1.2 it's possible to use UDP transport for graphite - I think it can be good workaround if you can listen on UDP port on your graphite installation thanks, maciek On 01/02/2017 13:22, Philipp Bussche wrote: Hi there, after moving my graphite service to another host my ta

Re: Graphite reporter recover from broken pipe

2017-02-01 Thread Jared Stehler
It appears that this change is only included in the as-yet-to-be-released metrics-graphite 3.1.3. -- Jared Stehler Chief Architect - Intellify Learning o: 617.701.6330 x703 > On Feb 1, 2017, at 7:22 AM, Philipp Bussche wrote: > > Hi there, > after moving my graphite service to another host m

Externalized Checkpoints vs Periodic Checkpoints

2017-02-01 Thread Yassine MARZOUGUI
Hi all, Could someone clarify the difference between externalized checkpoints[1] and regular periodic checkpoints[2]? Moreover, I have a question regarding the retention of checkpoints: For regular checkpoints, does the last checkpoint discard the previous ones? If yes, is that the case too for th

Graphite reporter recover from broken pipe

2017-02-01 Thread Philipp Bussche
Hi there, after moving my graphite service to another host my task manager does not recover monitoring and continues to complain about a broken pipe issue. It sounds a bit like this one: https://github.com/dropwizard/metrics/pull/1036 What do I need to do to update dropwizard to a 3.1.x version to

Apache Atlas Bridge for Flink

2017-02-01 Thread Stefan Papp
Hi, Is an Apache Atlas Bridge for Flink available? Has this feature already been requested. Stefan

Re: Cannot cancel job with savepoint due to timeout

2017-02-01 Thread Bruno Aranda
Maybe, though it could be good to be able to override in the command line somehow, though I guess I could just change the flink config. Many thanks Yuri, Bruno On Wed, 1 Feb 2017 at 07:40 Yury Ruchin wrote: > Hi Bruno, > > From the code I conclude that "akka.client.timeout" setting is what > a