Partitions and the number of cores/executors

2019-03-13 Thread mbilalce . dev
Hi, I am working with Gelly graph library but I think the question is applicable in general. I just want to confirm if a single data partition in Flink is executed by only a single executor/core? i.e. multiple executors can't be utilized to process a single partition in parallel. So, if I need

Custom Partition/Task Placement Algorithm

2019-03-11 Thread mbilalce . dev
Hi, I am experimenting with Gelly and Graph partitioning. I wanted to perform some experiments where I can control the placement of partitions of the graph. As far as I know there is no out of the box way of implementing such a capability in Flink. So I am looking for guidance regarding the c