Hi,
Flink distributes task instances to slots and does not expose physical
machines.
Records are partitioned to task instances by hash partitioning. It is also
not possible to guarantee that the records in two different operators are
send to the same slot.
Sharing information by side-passing it (e.g., via a file on a machine or in
a static object) is an anti-pattern and should be avoided.
Best, Fabian
2018-06-24 20:52 GMT+02:00 Vijay Balakrishnan <bvija...@gmail.com>:
> Hi,
>
> Need to partition by cameraWithCube.getCam() 1st using
> parallelCamTasks(passed in as args).
>
> Then within each partition, need to partition again by
> cameraWithCube.getTs() but need to make sure each of the 2nd partition by
> getTS() runs on the same physical node ?
>
> How do I achieve that ?
>
> DataStream<CameraWithCube> cameraWithCubeDataStream = env
> .addSource(new Source(....))
> .keyBy((cameraWithCube) -> cameraWithCube.getCam() )
> .process(new ProcessFunction<CameraWithCube, CameraWithCube>() {
> public void processElement(CameraWithCube cameraWithCube,
> Context context, Collector<CameraWithCube> collector) throws Exception {
> //do nothing
> }
> })
> .slotSharingGroup("camSharingGroup")//TODO: how to add camera# of
> the partition
> .setParallelism(parallelCamTasks)
> .keyBy((cameraWithCube) -> cameraWithCube.getTs())
> .process(new ProcessFunction<CameraWithCube, CameraWithCube>() {
> public void processElement(CameraWithCube cameraWithCube,
> Context context, Collector<CameraWithCube> collector) throws Exception {
> //TODO: process code
> }
> })
> .setParallelism(noOfSlotsInEachPhysicalNode)//TODO: how many
> parallel tasks within physical node
> .slotSharingGroup("??");//TODO: in same physical node
>
> TIA
>
