Re: Flink Batch Processing

2020-09-29 Thread Timo Walther
Piotrek [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 pon., 28 wrz 2020 o 15:14 s_penakalap...@yahoo.com <mailto:s_penakalap...@yahoo.com> mailto:s_penakalap...@yahoo.com>> napisał(a): Hi All, Need your help in Flink B

Re: Flink Batch Processing

2020-09-29 Thread Till Rohrmann
> Piotrek > > [1] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 > > pon., 28 wrz 2020 o 15:14 s_penakalap...@yahoo.com < > s_penakalap...@yahoo.com> napisał(a): > > Hi All, > > Need your help in Flink Batch processing: scenario described

Re: Flink Batch Processing

2020-09-29 Thread s_penakalap...@yahoo.com
could help here. Piotrek [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 pon., 28 wrz 2020 o 15:14 s_penakalap...@yahoo.com napisał(a): Hi All, Need your help in Flink Batch processing: scenario described below: we have multiple vehicles, we get data from each vehicl

Re: Flink Batch Processing

2020-09-28 Thread Piotr Nowojski
66741 pon., 28 wrz 2020 o 15:14 s_penakalap...@yahoo.com napisał(a): > Hi All, > > Need your help in Flink Batch processing: scenario described below: > > we have multiple vehicles, we get data from each vehicle at a very high > speed, 1 record per minute. > thresholds c

Flink Batch Processing

2020-09-28 Thread s_penakalap...@yahoo.com
Hi All, Need your help in Flink Batch processing: scenario described below: we have multiple vehicles, we get data from each vehicle at a very high speed, 1 record per minute.thresholds can be set by the owner for each vehicle.  Say: we have 3 vehicles, threshold is set for 2 vehicles. Vehicle 1

Re: Flink batch processing fault tolerance

2017-02-17 Thread Aljoscha Krettek
ready focusing on realizing the ideas mentioned in FLIP1, > wish to contirbute to flink in months. > > Best, > > Zhijiang > > -- > 发件人:Si-li Liu > 发送时间:2017年2月17日(星期五) 11:22 > 收件人:user > 主 题:Re: Flink ba

回复:Flink batch processing fault tolerance

2017-02-16 Thread wangzhijiang999
--发件人:Si-li Liu 发送时间:2017年2月17日(星期五) 11:22收件人:user 主 题:Re: Flink batch processing fault tolerance Hi,  It's the reason why I gave up use Flink for my current project and pick up traditional Hadoop Framework again.  2017-02-17 10:56 GMT+08:00 Renjie Liu : https://cwiki.apache.org/confluence/di

Re: Flink batch processing fault tolerance

2017-02-16 Thread Si-li Liu
a Krettek [mailto:aljos...@apache.org] >> *Sent:* Thursday, February 16, 2017 2:48 PM >> *To:* user@flink.apache.org >> *Subject:* Re: Flink batch processing fault tolerance >> >> >> >> Hi, >> >> yes, this is indeed true. We had some plans for

Re: Flink batch processing fault tolerance

2017-02-16 Thread Renjie Liu
gt; > > > > > *From:* Aljoscha Krettek [mailto:aljos...@apache.org] > *Sent:* Thursday, February 16, 2017 2:48 PM > *To:* user@flink.apache.org > *Subject:* Re: Flink batch processing fault tolerance > > > > Hi, > > yes, this is indeed true. We had some plans fo

RE: Flink batch processing fault tolerance

2017-02-16 Thread Anton Solovev
Hi Aljoscha, Could you share your plans of resolving it? Best, Anton From: Aljoscha Krettek [mailto:aljos...@apache.org] Sent: Thursday, February 16, 2017 2:48 PM To: user@flink.apache.org Subject: Re: Flink batch processing fault tolerance Hi, yes, this is indeed true. We had some plans for

Re: Flink batch processing fault tolerance

2017-02-16 Thread Aljoscha Krettek
Hi, yes, this is indeed true. We had some plans for how to resolve this but they never materialised because of the focus on Stream Processing. We might unite the two in the future and then you will get fault-tolerant batch/stream processing in the same API. Best, Aljoscha On Wed, 15 Feb 2017 at 0

Flink batch processing fault tolerance

2017-02-15 Thread Renjie Liu
Hi, all: I'm learning flink's doc and curious about the fault tolerance of batch process jobs. It seems that when one of task execution fails, the whole job will be restarted, is it true? If so, isn't it impractical to deploy large flink batch jobs? -- Liu, Renjie Software Engineer, MVAD

Re: Flink Batch Processing with Kafka

2016-08-03 Thread Prabhu V
If your environment is not kerberized (or if you can offord to restart the job every 7 days), a checkpoint enabled, flink job with windowing and the count trigger, would be ideal for your requirement. Check the api's on flink windows. I had something like this that worked stream.keyBy(0).countW

Flink Batch Processing with Kafka

2016-08-03 Thread Alam, Zeeshan
Hi, Flink works very well with Kafka if you wish to stream data. Following is how I am streaming data with Kafka and Flink. FlinkKafkaConsumer08 kafkaConsumer = new FlinkKafkaConsumer08<>(KAFKA_AVRO_TOPIC, avroSchema, properties); DataStream messageStream = env.addSource(kafkaConsumer); Is th