Actual byte-streams in multiple-node pipelines

2016-01-20 Thread Tal Maoz
Hey, I’m a new user to Flink and I’m trying to figure out if I can build a pipeline I’m working on using Flink. I have a data source that sends out a continues data stream at a bandwidth of anywhere between 45MB/s to 600MB/s (yes, that’s MiB/s, not Mib/s, and NOT a series of individual messages

Re: Actual byte-streams in multiple-node pipelines

2016-01-20 Thread Tal Maoz
..@gmail.com> wrote: > >> I think with sufficient processing power flink can do the above mentioned >> task using the stream api >> <https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/index.html> >> . >> >> Thanks, >> *Ritesh Kumar Singh,* &

Re: Actual byte-streams in multiple-node pipelines

2016-01-21 Thread Tal Maoz
>> a source that supports that. If you use a custom source, you may need a few >> lines to integrate it with the checkpoint mechanism, but that is very >> doable. >> >> Hope that helps! >> >> Greetings, >> Stephan >> >> >> >> &