Re: Benchmarking streaming frameworks

2017-03-23 Thread Dominik Safaric
Dear Giselle, Various stream processing engines benchmarks already exist. Here are only a few of them I believe are worthwhile mentioning: http://ieeexplore.ieee.org/document/7530084/ https://www.usenix.org/node/188989

Re: Benchmarking streaming frameworks

2017-03-23 Thread Michael Noll
A recent one is "Analytics on Fast Data: Main-Memory Database Systems versus Modern Streaming Systems" ( http://db.in.tum.de/~kipf/papers/fastdata.pdf) For the record, the paper above doesn't yet cover/realize that, nowadays, the Kafka project includes native stream processing capabilities aka the

Re: Benchmarking streaming frameworks

2017-03-23 Thread Felix Neutatz
Hi, our team already created a benchmark framework for batch processing (including MR,Yarn, Spark, Flink), maybe you like to extend it for streaming: https://github.com/peelframework/peel Best regards, Felix On Mar 23, 2017 11:51, "Christophe Salperwyck" wrote: Good idea! You could test Akka s

Re: Benchmarking streaming frameworks

2017-03-23 Thread Christophe Salperwyck
Good idea! You could test Akka streams too. Lots of documents exist: https://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at https://github.com/yahoo/streaming-benchmarks Cheers, Christophe 2017-03-23 11:09 GMT+01:00 Giselle van Dongen : > Dear users of Strea

Benchmarking streaming frameworks

2017-03-23 Thread Giselle van Dongen
Dear users of Streaming Technologies, As a PhD student in big data analytics, I am currently in the process of compiling a list of benchmarks (to test multiple streaming frameworks) in order to create an expanded benchmarking suite. The benchmark suite is being developed as a part of my current wo