Re: Performance and Latency Chart for Flink

Hi, I am not aware of periodic performance runs for the Flink releases. I know a few benchmarks which have been published at different points in time like [1], [2], and [3] (you'll probably find more). In general, fair benchmarks that compare different systems (if there is such thing) are very di

Re: Performance and Latency Chart for Flink

FYI, we, at a well known IT department, have been actively measuring Beam Flink Runner performance using MIT's Linear Road to stress the Flink Cluster servers.The results, thus far does not even come close to the previous streaming engines we have bench-marked.Our optimistic assumption was, when

[jira] [Created] (FLINK-4628) User class loader unavailable during input split assignment

Maximilian Michels created FLINK-4628: - Summary: User class loader unavailable during input split assignment Key: FLINK-4628 URL: https://issues.apache.org/jira/browse/FLINK-4628 Project: Flink

DataStream of Future

Hello, I just started learning Flink (using Scala) recently, and I developed a job that, in short, does this steps: - Reads json messages from Kafka - Enriches the messages, reading data from Cassandra (using Phantom DSL) - Puts the enriched messages back to ano

Re: Performance and Latency Chart for Flink

Hi Amir, it would be great if you could link to the details of your benchmark environment if you make such claims. Compared to which IBM system? Characteristics of your machines? Configuration of the software? Implementation code? etc. In general the Beam Runner also adds some overhead compa

Re: DataStream of Future

Hi Albert, you cannot use Futures between operators as objects are seralialized and possibly sent through the cluster immediatly. Right now there is no straight forward way in Flink to do async calls. However, there is a discussion going on which you might wanna join [1]. As far as I know, th

Re: DataStream of Future

Hi Timo, Thank you for the fast reply :) I will check and follow the discussion, it would be really cool to be able to have that feature. As for the updates, there I'm using Futures which update accumulators on success / failure, and everything seems to be working fine, I tested both in local mod

Re: DataStream of Future

Hi,Albert I think that the futures may be lost if the machine is down for some reasons. So simply using the futures may not be acceptable in production. I hope this is helpful. Thanks > 在 2016年9月16日，下午3:01，alghimo 写道： > > Hi Timo, > > Thank you for the fast reply :) I will check and follow th

Re: Contribution

Welcome Hasan! I also studied at FU Berlin. I would suggest to work on some of the issues you experience as a data engineer working with Flink. Even starting with something trivial like a one-liner or updating documentation would be great. Cheers, Max On Fri, Sep 9, 2016 at 1:40 PM, Hasan Gürcan

Re: Contribution

Welcome Hasan! This link might help you: http://flink.apache.org/how-to-contribute.html#contribute-code Timo Am 16/09/16 um 15:18 schrieb Maximilian Michels: Welcome Hasan! I also studied at FU Berlin. I would suggest to work on some of the issues you experience as a data engineer working w

Re: Performance and Latency Chart for Flink

Hi Fabian,FYI. This is report on other engines we did the same type of bench-marking.Also explains what Linear Road bench-marking is.Thanks for your help. http://www.slideshare.net/RedisLabs/walmart-ibm-revisit-the-linear-road-benchmark https://github.com/IBMStreams/benchmarks  https://www.datato

[jira] [Created] (FLINK-4629) Kafka v 0.10 Support

Mariano Gonzalez created FLINK-4629: --- Summary: Kafka v 0.10 Support Key: FLINK-4629 URL: https://issues.apache.org/jira/browse/FLINK-4629 Project: Flink Issue Type: Wish Reporte