Re: Flink's WordCount at scale of 1BLN of unique words

2016-05-31 Thread Xtra Coder
Thanks, things are clear so far.

Re: Alter Flink's execution graph at run-time

2016-05-31 Thread Xtra Coder
Thanks, altering via pause/update/resume is OK, at least for now. Will try it on practice. Just in case - question was inspired by Apache NiFi. If you haven't seen this https://www.youtube.com/watch?v=sQCgtCoZyFQ - at 29:10. I would say such thing is a must have feature in "production" where stopp

Alter Flink's execution graph at run-time

2016-05-30 Thread Xtra Coder
Hello, I'm curious about ability to alter processing of streams in Flink at run-time. Potential use-case may look like following: 1. I have a stream already running (i.e. data processing is already started) in the Flink's cluster 2. At some point of time I decide that I need to add some more st

Re: Flink's WordCount at scale of 1BLN of unique words

2016-05-24 Thread Xtra Coder
11:13 AM, Matthias J. Sax wrote: > Are you talking about a streaming or a batch job? > > You are mentioning a "text stream" but also say you want to stream 100TB > -- indicating you have a finite data set using DataSet API. > > -Matthias > > On 05/22/2016 09:

Flink's WordCount at scale of 1BLN of unique words

2016-05-22 Thread Xtra Coder
Hello, Question from newbie about how Flink's WordCount will actually work at scale. I've read/seen rather many high-level presentations and do not see more-or-less clear answers for following … Use-case: -- there is huuuge text stream with very variable set of words – let's say 1BLN