FLink Streaming - Parallelize ArrayList

2015-09-28 Thread defstat
Hi all, Is there a way to either Broadcast or Parallelize an ArrayList of custom objects? The application is as follows: I have 2 streams taken from two sockets. The first stream contains vectors that should populate one list of vectors by deciding whether an arriving vector is part of the lis

Re: Apache Flink streaming features summary

2015-09-28 Thread Flavio Pompermaier
Ok, thanks for the pointers Stephan! Best, Flavio On Mon, Sep 28, 2015 at 4:56 PM, Sebastian wrote: > These marketing-style comparisons are almost always either wrong, or >> heavily tainted to make one particular project look good. They are >> pretty bad style for Apache projects, in my opinion

Re: Apache Flink streaming features summary

2015-09-28 Thread Sebastian
These marketing-style comparisons are almost always either wrong, or heavily tainted to make one particular project look good. They are pretty bad style for Apache projects, in my opinion. For that reason, we try to make more technical statements and comparisons. +1

Re: Apache Flink streaming features summary

2015-09-28 Thread Stephan Ewen
Hi Flavio! These marketing-style comparisons are almost always either wrong, or heavily tainted to make one particular project look good. They are pretty bad style for Apache projects, in my opinion. For that reason, we try to make more technical statements and comparisons. There are quite some p

Re: JVM core dump running Flink streaming

2015-09-28 Thread Till Rohrmann
When the CMS garbage collector is used we additionally set -XX:+CMSClassUnloadingEnabled. Everything else are the default values. ​ On Mon, Sep 28, 2015 at 11:22 AM, Jakob Ericsson wrote: > Yes. I just wanted to know if you have any preferred settings for CMS or > should I use fairly standard co

Re: JVM core dump running Flink streaming

2015-09-28 Thread Jakob Ericsson
Yes. I just wanted to know if you have any preferred settings for CMS or should I use fairly standard configs? On Mon, Sep 28, 2015 at 11:19 AM, Till Rohrmann wrote: > Hi Jakob, > > you can control the used garbage collector by specifying the env.java.opts > configuration parameter in your flink

Re: JVM core dump running Flink streaming

2015-09-28 Thread Till Rohrmann
Hi Jakob, you can control the used garbage collector by specifying the env.java.opts configuration parameter in your flink-conf.yaml. Here you can basically specify all the JVM options which will be given to the JVMs. Thus, in your case you could try the following settings: env.java.opts: -XX:+Use

Re: Apache Flink streaming features summary

2015-09-28 Thread Flavio Pompermaier
sorry, I forgot to paste it :( https://www.datatorrent.com/project-apex/ On Mon, Sep 28, 2015 at 10:10 AM, Fabian Hueske wrote: > Can you share a link to the comparison? > > Thanks, Fabian > > 2015-09-28 9:32 GMT+02:00 Flavio Pompermaier : > >> Hi to all, >> >> I saw this comparison vs Spark, S

JVM core dump running Flink streaming

2015-09-28 Thread Jakob Ericsson
Hi, I'm testing Flink streaming but seems to have some problems with jvm core dumps. I haven't really looked at the heap dump yet. It seems to be related to the G1 GC. If I want to go back to CMS-GC, do you have any preferred settings? # # A fatal error has been detected by the Java Runtime E

Re: Apache Flink streaming features summary

2015-09-28 Thread Fabian Hueske
Can you share a link to the comparison? Thanks, Fabian 2015-09-28 9:32 GMT+02:00 Flavio Pompermaier : > Hi to all, > > I saw this comparison vs Spark, Storm and Apache Apex. > Since I didn't have time to look at the streaming part of Apache Flink, it > could be interesting to have such a table (

Apache Flink streaming features summary

2015-09-28 Thread Flavio Pompermaier
Hi to all, I saw this comparison vs Spark, Storm and Apache Apex. Since I didn't have time to look at the streaming part of Apache Flink, it could be interesting to have such a table (at least for the streaming features) also for Flink documented somewhere..what do you think? Best, Flavio