Re: Flink multithreading, CoFlatMapFunction, CoProcessFunction, internal state

2017-08-21 Thread Nico Kruber
Hi Chao, what I meant by "per-record base" was actually supposed to be "per-event base" (event = one entity of whatever the stream contains). As from the API: processing is supposed to be one event at a time and this is what is performed internally, too. Nico On Thursday, 17 August 2017 05:06:

Re: Flink multithreading, CoFlatMapFunction, CoProcessFunction, internal state

2017-08-16 Thread Chao Wang
Thank you! Nico. That helps me a lot! 2a) That really clarifies my understanding about Flink. Yes, I think I have used static references, since I invoked a native function (implemented through JNI) which I believe only has one instance per process. And I guess the reason why those Java synchro

Re: Flink multithreading, CoFlatMapFunction, CoProcessFunction, internal state

2017-08-16 Thread Nico Kruber
Hi Chao, 1) regarding the difference of CoFlatMapFunction/CoProcessFunction, let me quote the javadoc of the CoProcessFunction: "Contrary to the {@link CoFlatMapFunction}, this function can also query the time (both event and processing) and set timers, through the provided {@link Context}. Wh

Flink multithreading, CoFlatMapFunction, CoProcessFunction, internal state

2017-08-14 Thread Chao Wang
Hi, I'd like to know if CoFlatMapFunction/CoProcessFunction is thread-safe, and to what extent? What's the difference between the two Functions? and in general, how does Flink prevent race conditions? Here's my case: I tried to condition on two input streams and produce the third stream if t