Hi, what are the artifacts that I have to import on maven in order to use Blink Api?
I am using Flink 1.8.0 and I am trying to import blink code to use its ExecutionContext <https://github.com/apache/flink/blob/master/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/context/ExecutionContext.java>. I want to do this in order to implement my own operator like it is implemented here <https://github.com/apache/flink/blob/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/runtime/bundle/BundleOperator.java>. I guess if I import flink-table everything should come inside the same jar as it is done here <https://github.com/apache/flink/blob/master/flink-table/flink-table-runtime-blink/pom.xml>. However, I cannot import "flink-table-runtime-blink". Eclipse says that it is a missing artifact. <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-table-planner_2.11</artifactId> <version>1.8.0</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-table-api-java-bridge_2.11</artifactId> <version>1.8.0</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-scala_2.11</artifactId> <version>1.8.0</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-table-common</artifactId> <version>1.8.0</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-table</artifactId> <version>1.8.0</version> <type>pom</type> <scope>provided</scope> </dependency> <dependency> <!-- THIS IS NOT POSSIBLE TO IMPORT --> <groupId>org.apache.flink</groupId> <artifactId>flink-table-runtime-blink</artifactId> <version>1.8.0</version> </dependency> *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com <https://felipeogutierrez.blogspot.com>* On Mon, Apr 15, 2019 at 9:49 AM Felipe Gutierrez < [email protected]> wrote: > Cool, thanks Kurt! > *-* > *- Felipe Gutierrez* > > *- skype: felipe.o.gutierrez* > *- **https://felipeogutierrez.blogspot.com > <https://felipeogutierrez.blogspot.com>* * > <https://felipeogutierrez.blogspot.com>* > > > On Mon, Apr 15, 2019 at 6:06 AM Kurt Young <[email protected]> wrote: > >> Hi, >> >> You can checkout the bundle operator which used in Blink to perform >> similar thing you mentioned: >> https://github.com/apache/flink/blob/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/runtime/bundle/BundleOperator.java >> >> Best, >> Kurt >> >> >> On Fri, Apr 12, 2019 at 8:05 PM Felipe Gutierrez < >> [email protected]> wrote: >> >>> Hi, >>> >>> I was trying to implement a better way to handle data skew using Flink >>> and I found this talk from #FlinkForward SF 2017: "Cliff Resnick & Seth >>> Wiesman - From Zero to Streaming <https://youtu.be/mSLesPzWplA?t=835>" >>> [1] which says that they used OneInputStreamOperator [2]. Through it, they >>> could implement the "combiner" in Hadoop (execute part of the reduce tasks >>> on the Map phase, before shuffling). >>> >>> I need some help here. What are some of the Flink source-code operators >>> that I can peek up to implement my on operator that deals with data skew? >>> Or maybe, is there someone that have an example of a use case similar to >>> this? >>> >>> [1] https://youtu.be/mSLesPzWplA?t=835 >>> [2] >>> https://ci.apache.org/projects/flink/flink-docs-master/api/java/index.html?org/apache/flink/streaming/api/functions/source/ContinuousFileReaderOperator.html >>> >>> Thanks! >>> Felipe >>> >>> *--* >>> *-- Felipe Gutierrez* >>> >>> *-- skype: felipe.o.gutierrez* >>> *--* *https://felipeogutierrez.blogspot.com >>> <https://felipeogutierrez.blogspot.com>* >>> >>
