Re: Different result on running Flink in local mode and Yarn cluster

2018-04-26 Thread Michael Latta
Not knowing the library or the config needs I do not have a suggestion. If the config is accumulated from inputs and needs to see all inputs I would suggest setting parallelism to 1 as an experiment, but it would need a redesign to run in parallel. Michael Sent from my iPad > On Apr 26, 2018

Re: Different result on running Flink in local mode and Yarn cluster

2018-04-25 Thread Michael Latta
Only the anonymous FlatMapFunction instance is sent to the TaskManager. Move the static field to that class. Michael Sent from my iPad > On Apr 25, 2018, at 10:42 PM, Soheil Pourbafrani > wrote: > > I run a code using Flink Java API that gets some bytes from Kafka and parses > it following

Re: Different result on running Flink in local mode and Yarn cluster

2018-04-25 Thread Jörn Franke
The problem maybe that it is still static. How will the parser use this HashMap? > On 26. Apr 2018, at 06:42, Soheil Pourbafrani wrote: > > I run a code using Flink Java API that gets some bytes from Kafka and parses > it following by inserting into Cassandra database using another library > s

Different result on running Flink in local mode and Yarn cluster

2018-04-25 Thread Soheil Pourbafrani
I run a code using *Flink* Java API that gets some bytes from *Kafka* and parses it following by inserting into *Cassandra* database using another library *static* method (both parsing and inserting results is done by the library). Running code on local in IDE, I get the desired answer, but running