Today I am happy to announce a prototype of an Apache Storm based backend for Apache Flink called S-link-y. Storm offers a number of advantages over Flink including enterprise level security and a distributed cache with in place updating. The Flink front end offers both batch processing APIs and an Apache BEAM implementation. As part of our testing we were surprised to discover that running the Storm WordCountTopology through the Flink-Storm compatibility layer on the S-link-y backend had 89% latency and 3x the throughput of running it on Storm itself and 78% the latency of a native Flink WordCount. To see the benchmark results, download the source code and try it out yourself please go to http://bit.ly/1gho7il We would love to begin discussions between the Flink and Storm PMCs about merging the two projects.
- Bobby
