Thanks for the link, I had experimented with those options, apart from
taskmanager.memory.off-heap: true. Turns out that allows it to run through
happily! I don't know if that is a peculiarity of a windows JVM, as I
understand that setting is purely an efficiency improvement?
For your first ques
Hi Vasia,
It seems to work OK up to about 50MB of input, and dies after that point.
If i disable just this connected components step the rest of my program is
happy with the full 1.5GB test dataset. It seems to be specifically limited
to GraphAlgorithms in my case.
Do you know what the units ar
Thanks Vasia,
Apologies, yes by workers i mean I have set taskmanager.numberOfTaskSlots: 8
and parallelism.default: 8 in flink-conf.yaml. I have also set
taskmanager.heap.mb: 6500
In the dashboard it is showing free memory as 5.64GB and Flink Managed
Memory as 3.90GB.
Thanks,
Rob
--
View this
Hi to all,
I’m aware there are a few threads on this, but I haven’t been able to solve
an issue I am seeing and hoped someone can help. I’m trying to run the
following:
val connectedNetwork = new org.apache.flink.api.scala.DataSet[Vertex[Long,
Long]](
Graph.fromTuple2DataSet(inputEdges, vertex
