Hi,
I think you are looking for this answer from David:
https://stackoverflow.com/questions/69799181/flink-streaming-do-the-events-get-distributed-to-each-task-slots-separately-acc
I think then you could technically create your partitioner - though
little bit cubersome - by mapping your existing keys to new keys who
will have then an output to the desired
group & slot.
Hope this may help,
Dario
On 04.03.22 23:54, Ken Krugler wrote:
Hi all,
I need to be able to control which slot a keyBy group goes to, in
order to compensate for a badly skewed dataset.
Any recommended approach to use here?
Previously (with a DataSet) I used groupBy followed by a
withPartitioner, and provided my own custom partitioner.
I posted this same question to
https://stackoverflow.com/questions/71357833/equivalent-of-dataset-groupby-withpartitioner-for-datastream
Thanks,
— Ken
--------------------------
Ken Krugler
http://www.scaleunlimited.com
Custom big data solutions
Flink, Pinot, Solr, Elasticsearch
