Re: Delta iteration not spilling to disk

Hi Joshua, I am running into a similar problem. Can you explain your solution a bit more ? A code snippet will help. Thanks -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Delta iteration not spilling to disk

avoid them from being mutated by the user-code JoinFunction. The stacktrace hints that the NPE is caused by copying a null record. This would happen if the solution set would not contain the key. I was not sure if there is a restriction of the delta iteration that all keys must be present in the i

Re: Delta iteration not spilling to disk

key. I was not sure if there is a restriction of the delta iteration that all keys must be present in the initial solution set. I tried to find this in the documentation but didn't see information on that. So I checked and was able to reproduce the problem. It is only possible to join the sol

Re: Delta iteration not spilling to disk

he solution set to unManaged (DeltaIteration.setSolutionSetUnManaged(true)). Best, Fabian 2017-10-24 21:09 GMT+02:00 Joshua Griffith mailto:jgriff...@campuslabs.com>>: I’m currently using a delta iteration within a batch job and received the following error: java.lang.RuntimeExceptio

Re: Delta iteration not spilling to disk

naged (DeltaIteration.setSolutionSetUnManaged(true)). Best, Fabian 2017-10-24 21:09 GMT+02:00 Joshua Griffith : > I’m currently using a delta iteration within a batch job and received the > following error: > > java.lang.RuntimeException: Memory ran out. Compaction failed. > numPartitions: 32 minPartit

Delta iteration not spilling to disk

I’m currently using a delta iteration within a batch job and received the following error: java.lang.RuntimeException: Memory ran out. Compaction failed. numPartitions: 32 minPartition: 11 maxPartition: 24 number of overflow segments: 0 bucketSize: 125 Overall memory: 23232512 Partition memory

delta iteration

Hello all, I need iteration number in delta iteration (or any kind of counter). Is there anyway to implement or extract it? Cheers, Alieh

Re: delta iteration

Hi Alieh, From a rich function call getIterationRuntimeContext().getSuperstepNumber() Greg > On Jul 12, 2017, at 9:56 AM, Alieh wrote: > > Hello all, > > I need iteration number in delta iteration (or any kind of counter). Is there > anyway to implement or extra

delta iteration

Hello all, I need iteration number in delta iteration (or any kind of counter). Is there anyway to implement or extract it? Cheers, Alieh

delta iteration

Hello all, I need iteration number in delta iteration (or any kind of counter). Is there anyway to implement or extract it? Cheers, Alieh

Re: modify solution set within the delta iteration

gt; > I’m doing a delta iteration in which my solution set in something like: > DataSet> > > Depending on a different dataset that i retrieve during the iteration, i > need to update the hashMap of the solution set in order to keep it up to > date. > > But i noticed that

Re: Intermediate solution set of delta iteration

sets from > every delta iteration? I tried union but the compiler gave me the error: > Exception in thread "main" > org.apache.flink.api.common.InvalidProgramException: Error: The only > operations allowed on the solution set are Join and CoGroup. > > > Best, > Mengq

modify solution set within the delta iteration

I try to explain my situation I’m doing a delta iteration in which my solution set in something like: DataSet> Depending on a different dataset that i retrieve during the iteration, i need to update the hashMap of the solution set in order to keep it up to date. But i noticed that i can

Intermediate solution set of delta iteration

Hi all, Are there any approaches here I could get intermediate solution sets from every delta iteration? I tried union but the compiler gave me the error: Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: Error: The only operations allowed on the soluti