Hi, You can checkout the bundle operator which used in Blink to perform similar thing you mentioned: https://github.com/apache/flink/blob/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/runtime/bundle/BundleOperator.java
Best, Kurt On Fri, Apr 12, 2019 at 8:05 PM Felipe Gutierrez < felipe.o.gutier...@gmail.com> wrote: > Hi, > > I was trying to implement a better way to handle data skew using Flink and > I found this talk from #FlinkForward SF 2017: "Cliff Resnick & Seth > Wiesman - From Zero to Streaming <https://youtu.be/mSLesPzWplA?t=835>" > [1] which says that they used OneInputStreamOperator [2]. Through it, they > could implement the "combiner" in Hadoop (execute part of the reduce tasks > on the Map phase, before shuffling). > > I need some help here. What are some of the Flink source-code operators > that I can peek up to implement my on operator that deals with data skew? > Or maybe, is there someone that have an example of a use case similar to > this? > > [1] https://youtu.be/mSLesPzWplA?t=835 > [2] > https://ci.apache.org/projects/flink/flink-docs-master/api/java/index.html?org/apache/flink/streaming/api/functions/source/ContinuousFileReaderOperator.html > > Thanks! > Felipe > > *--* > *-- Felipe Gutierrez* > > *-- skype: felipe.o.gutierrez* > *--* *https://felipeogutierrez.blogspot.com > <https://felipeogutierrez.blogspot.com>* >
