Cool, thanks Kurt! *-* *- Felipe Gutierrez* *- skype: felipe.o.gutierrez* *- **https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* * <https://felipeogutierrez.blogspot.com>*
On Mon, Apr 15, 2019 at 6:06 AM Kurt Young <ykt...@gmail.com> wrote: > Hi, > > You can checkout the bundle operator which used in Blink to perform > similar thing you mentioned: > https://github.com/apache/flink/blob/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/runtime/bundle/BundleOperator.java > > Best, > Kurt > > > On Fri, Apr 12, 2019 at 8:05 PM Felipe Gutierrez < > felipe.o.gutier...@gmail.com> wrote: > >> Hi, >> >> I was trying to implement a better way to handle data skew using Flink >> and I found this talk from #FlinkForward SF 2017: "Cliff Resnick & Seth >> Wiesman - From Zero to Streaming <https://youtu.be/mSLesPzWplA?t=835>" >> [1] which says that they used OneInputStreamOperator [2]. Through it, they >> could implement the "combiner" in Hadoop (execute part of the reduce tasks >> on the Map phase, before shuffling). >> >> I need some help here. What are some of the Flink source-code operators >> that I can peek up to implement my on operator that deals with data skew? >> Or maybe, is there someone that have an example of a use case similar to >> this? >> >> [1] https://youtu.be/mSLesPzWplA?t=835 >> [2] >> https://ci.apache.org/projects/flink/flink-docs-master/api/java/index.html?org/apache/flink/streaming/api/functions/source/ContinuousFileReaderOperator.html >> >> Thanks! >> Felipe >> >> *--* >> *-- Felipe Gutierrez* >> >> *-- skype: felipe.o.gutierrez* >> *--* *https://felipeogutierrez.blogspot.com >> <https://felipeogutierrez.blogspot.com>* >> >
