RE: the design of spilling to disk

2017-09-20 Thread Newport, Billy
o:k.klou...@data-artisans.com>> Sent: Tuesday, September 19, 2017 18:10 To: Florin Dinu Cc: user@flink.apache.org<mailto:user@flink.apache.org>; fhue...@apache.org<mailto:fhue...@apache.org> Subject: Re: the design of spilling to disk Hi Florin, Unfortunately,

Re: the design of spilling to disk

2017-09-19 Thread Kurt Young
oudas > *Sent:* Tuesday, September 19, 2017 18:10 > *To:* Florin Dinu > *Cc:* user@flink.apache.org; fhue...@apache.org > *Subject:* Re: the design of spilling to disk > > Hi Florin, > > Unfortunately, there is no design document. > > The UnilateralSortMerger.java is used

Re: the design of spilling to disk

2017-09-19 Thread Florin Dinu
From: Kostas Kloudas Sent: Tuesday, September 19, 2017 18:10 To: Florin Dinu Cc: user@flink.apache.org; fhue...@apache.org Subject: Re: the design of spilling to disk Hi Florin, Unfortunately, there is no design document. The UnilateralSortMerger.java is used in the batch

Re: the design of spilling to disk

2017-09-19 Thread Kostas Kloudas
Hi Florin, Unfortunately, there is no design document. The UnilateralSortMerger.java is used in the batch processing mode (not is streaming) and, in fact, the code dates some years back. I cc also Fabian as he may have more things to say on this. Now for the streaming side, Flink uses 3 state