Re: Intermediate Data Caching

2016-07-19 Thread Saliya Ekanayake
Thank you, Ufuk! On Tue, Jul 19, 2016 at 5:51 AM, Ufuk Celebi wrote: > PS: I forgot to mention that also constant iteration input is cached. > > On Mon, Jul 18, 2016 at 11:27 AM, Ufuk Celebi wrote: > > Hey Saliya, > > > > the result of each iteration (super step) that is fed back to the > > ite

Re: Intermediate Data Caching

2016-07-19 Thread Ufuk Celebi
PS: I forgot to mention that also constant iteration input is cached. On Mon, Jul 18, 2016 at 11:27 AM, Ufuk Celebi wrote: > Hey Saliya, > > the result of each iteration (super step) that is fed back to the > iteration is cached. For the iterate operator that is the last partial > solution and fo

Re: Intermediate Data Caching

2016-07-18 Thread Ufuk Celebi
Hey Saliya, the result of each iteration (super step) that is fed back to the iteration is cached. For the iterate operator that is the last partial solution and for the delta iterate operator it's the current solution set (https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/batch/it

Intermediate Data Caching

2016-07-17 Thread Saliya Ekanayake
Hi, I am trying to understand what's the intermediate caching support in Flink. For example, when there's an iterative dataset what's being cached between iterations. Is there some documentation on this? Thank you, Saliya -- Saliya Ekanayake Ph.D. Candidate | Research Assistant School of Inform