Hi Joshua,
I am running into a similar problem. Can you explain your solution a bit
more ? A code snippet will help.
Thanks
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Hi Fabian,
Switching the solution set join to a co-group indeed fixed the issue. Thank you!
Joshua
On Oct 25, 2017, at 11:00 AM, Fabian Hueske
mailto:fhue...@gmail.com>> wrote:
Hi Joshua,
with the unmanaged solution set, the records are not serialized but they need
to be copied to avoid them
Hi Joshua,
with the unmanaged solution set, the records are not serialized but they
need to be copied to avoid them from being mutated by the user-code
JoinFunction.
The stacktrace hints that the NPE is caused by copying a null record. This
would happen if the solution set would not contain the ke
Hello Fabian,
Thank you for your response. I tried setting the solution set to unmanaged and
got a different error:
2017-10-24 20:46:11.473 [Join (join solution trees) (1/8)] ERROR
org.apache.flink.runtime.operators.BatchTask - Error in task code: Join (join
solution trees) (1/8)
java.lang.N
Hi Joshua,
that is correct. Delta iterations cannot spill to disk. The solution set is
managed in an in-memory hash table.
Spilling that hash table to disk would have a significant impact on the
performance.
By default the hash table is organized in Flink's managed memory.
You can try to increase
I’m currently using a delta iteration within a batch job and received the
following error:
java.lang.RuntimeException: Memory ran out. Compaction failed. numPartitions:
32 minPartition: 11 maxPartition: 24 number of overflow segments: 0 bucketSize:
125 Overall memory: 23232512 Partition memory:
