答复：答复：how to understand the flink flow control

Hi Ufuk,           Thank you for your instant reply and I have got the basic idea.             I am researching flink code now, and  if I have further specific questions next time, I will send to dev@flink.a.o . Best wishes, Zhijiang Wang ---

spark vs flink low memory available

hi community, i have build a spark and flink k-means application. my test case is a clustering on 1 million points on 3node cluster. in memory bottlenecks begins flink to outsource to disk and work slowly but works. however spark lose executers if the memory is full and starts again (infinety loo

Re: Discussion: two different computation approaches for flink and spark(Operator-centric, intermediate data-centric)

Hi! The terms are not yet really standard terms, we only use them to explain the differences in the runtime design of the two systems. Maybe they will sink into the data processing vocabulary... First and most importantly, the two are not mutually exclusive. Both have to deal with computation and

Re: Invalid argument reading a file containing a Kryo object

Thanks a lot! 2015-08-10 12:20 GMT+02:00 Flavio Pompermaier : > Done through https://issues.apache.org/jira/browse/FLINK-2503 > > Thanks again, > Flavio > > On Mon, Aug 10, 2015 at 12:11 PM, Fabian Hueske wrote: > >> Congrats that you got your InputFormat working! >> It is true, there can be a f

Re: Invalid argument reading a file containing a Kryo object

Done through https://issues.apache.org/jira/browse/FLINK-2503 Thanks again, Flavio On Mon, Aug 10, 2015 at 12:11 PM, Fabian Hueske wrote: > Congrats that you got your InputFormat working! > It is true, there can be a few inconsistencies in the Formats derived from > FileInputFormat. > > It woul

Re: Invalid argument reading a file containing a Kryo object

Congrats that you got your InputFormat working! It is true, there can be a few inconsistencies in the Formats derived from FileInputFormat. It would be great if you could open JIRAs for these issues. Otherwise, the might get lost on the mailing list. Thanks, Fabian 2015-08-10 12:02 GMT+02:00 Fla

Re: Invalid argument reading a file containing a Kryo object

Hi Fabian, thanks to your help I finally managed to successfully generate a DataSet from my folder but I think that there are some inconsistencies in the hierarchy of InputFormats. The *BinaryOutputFormat*/*TypeSerializerInputFormat* should somehow inherit the behaviour of the FileInputFormat (so r

Re: 答复：how to understand the flink flow control

Good to hear. Answers to your questions are inline. I think we should move the discussion to the dev@flink.a.o list though if there are more questions. On Mon, Aug 10, 2015 at 9:29 AM, wangzhijiang999 wrote: > Hi Ufuk, > > >Thank you for your detail and clear explaination! I reviewed t

答复：how to understand the flink flow control

Hi Ufuk,        Thank you for your detail and clear explaination!  I reviewed the code based on your info and got it clearly. For local transfer: When producer emits results to ResultPartition, it will request  Buffer from the pool. If there  are no available Buffer, it will wait .The ResultPart