Re: Latest spark yahoo benchmark

2017-06-26 Thread nragon
Yes, indeed. That's why we choose Flink instead all the others. This post was just pure curiosity to see spark trying to migrate into a pure streaming engine. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Latest-spark-yahoo-benchmark-tp13

Re: Latest spark yahoo benchmark

2017-06-26 Thread Stephan Ewen
@nragon - I think this is a classical "benchmarketing" post. A few thoughts on that - Everyone can tune their system to be best. We ran Flink with even higher throughput than that: https://image.slidesharecdn.com/benchmark-mapr-160407212254/95/extending-the-yahoo-streaming-benchmark-mapr-benchma

Re: Latest spark yahoo benchmark

2017-06-18 Thread Matthias J. Sax
From my understanding, the benchmark was done using Structured Streaming that is still based on micro batching. There are not throughput numbers for the new "Continuous Processing" model Spark want to introduce. Only some latency numbers. Also note, that the new "Continuous Processing" will not gi