[Survey] How many people use Flink with AWS Kinesis sink

Hi guys, We want to have a more accurate idea of how many people are writing Flink's computation result to AWS Kinesis, and how many people had successful Flink deployment against Kinesis? The reason I ask for the survey is because we have been trying to make our Flink jobs and Kinesis sin

Re: Memory Issue

One would need to look at your code and possible on some heap statistics. Maybe something wrong happens when you cache them (do you use a 3rd party library or your own implementation?). Do you use a stable version of your protobuf library (not necessarily the most recent). You also may want to l

Re:Re: Re: Just do a survey, how many people give up the storm and turn to Flink ?

Thanks, Renjie. Same here. minglei At 2017-08-21 11:19:48, "Renjie Liu" wrote: We are running both systems. Old existing systems are still running on storm and new jobs are all running on flink. On Mon, Aug 21, 2017 at 3:06 AM kaniska Mandal wrote: Storm supports 'exactly once' - ht

Memory Issue

Hi, I have a pipeline running on flink which ingests around 6k messages per second. Each message is around 1kb and it passes through various stages like filter, 5 sec tumbling window per key etc.. and finally flatmap to computation before sending it to kafka sink. The data is first ingested as pro

Re: Re: Just do a survey, how many people give up the storm and turn to Flink ?

We are running both systems. Old existing systems are still running on storm and new jobs are all running on flink. On Mon, Aug 21, 2017 at 3:06 AM kaniska Mandal wrote: > Storm supports 'exactly once' - > http://storm.apache.org/releases/current/Trident-tutorial.html > > Storm supports 'statefu

Flink HA with Kubernetes, without Zookeeper

Hi, I am new to Flink and trying to bring up a Flink cluster on top of Kubernetes. For HA setup, with kubernetes, I think I just need one job manager and do not need Zookeeper? I will store all states to S3 buckets. So in case of failure, kubernetes can just bring up a new job manager without losi

Re: [ANNOUNCE] Flink Forward Berlin (11-13 Sep 2017) Call for Submissions is open now

Dear Flink Community, I'd like to quickly bring your attention to Flink Forward Berlin again. The conference is happening in Berlin, Germany from September 11-13. There's a pretty impressive list of talks, including - keynotes from Alibaba, data Artisans, ING -

Re: Re: Just do a survey, how many people give up the storm and turn to Flink ?

Storm supports 'exactly once' - http://storm.apache.org/releases/current/Trident-tutorial.html Storm supports 'stateful processing' - http://storm.apache.org/releases/2.0.0-SNAPSHOT/State-checkpointing.html Storm is blazing fast - when its properly configured to use distributed (via some client l

Re:Re: Just do a survey, how many people give up the storm and turn to Flink ?

If you combine Storm with Redis for managing state you still do not have "exactly once" when a failure of a processing node occurs. With Flink you do have that. Niels On 19 Aug 2017 12:00, "mingleizhang" <18717838...@163.com> wrote: > Thanks Niels for your answer. Yes, storm does not support sta