Re: Related datastream

2017-06-22 Thread nragon
I believe I could try with microbatch system in order to release some memory. Meaning, if I have to generate 1M records splitting in 100m each iteration. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Related-datastream-tp13901p13908.html Se

Re: Related datastream

2017-06-21 Thread nragon
The reason I'm doing it on stream is because i can have many records in memory and I want to execute this in an ordinary laptop. With streaming i can achieve this. So i set my links between a and c with 0..4 meaning each record from a can have between 0 or 4 records, same for b. But for now leta co

Re: Related datastream

2017-06-21 Thread Jonas Gröger
Hi nragon, apparently I didn't read the P.S. since I assumed its not important. Silly me. So you are trying to join stream A and B to stream C with stream A and B being keyed. Alright. Are how often do matching elements (matched by primary key) from A and B arrive on your operator to-be-implement

Re: Related datastream

2017-06-21 Thread Jonas Gröger
Hey nragon! Do the two streams A and B have some sort of id or key or how do you plan on joining them? Do you just want to join A and B with elements a and b as they arrive (one in state and join with the next arriving one from the other stream)? >From what you are asking, this should be no probl