Flink Table & SQL doesn't work in very simple example

2016-06-21 Thread Jark Wu
Hi, I’m trying to use Flink Table 1.1-SNAPSHOT where I want to use Table API and SQL in my project. But when I run the very simple example WordCountTable, I encountered the following exception : Exception in thread "main" java.lang.NoSuchMethodError: org.apache.calcite.rel.logical.LogicalAgg

Re: UDP support in Streaming API

2016-06-21 Thread Sarvani
Hi Janani, Have you implemented UDP source using the SourceFunction interface? -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/UDP-support-in-Streaming-API-tp4715p12182.html Sent from the Apache Flink Mailing List archive. mailing list archive at

Re: Kafka doesn't work in very simple test

2016-06-21 Thread Robert Metzger
Hi Adrian, you need to provide a bit more information than this to understand the problem. What are the taskmanager logs saying? How do you test if data is being read from Kafka or not? On Fri, Jun 17, 2016 at 11:36 AM, Adrian Portabales < adrianportaba...@gmail.com> wrote: > Other jobs works fi

Re: Advice - Drools in Flink

2016-06-21 Thread Aljoscha Krettek
Hi, you can also take a look at this: https://techblog.king.com/rbea-scalable-real-time-analytics-king/. From a high level it seems Drools implements something similar to the system that they developed on top of Flink. Cheers, Aljoscha On Tue, 21 Jun 2016 at 12:04 Robert Metzger wrote: > Hi Ant

Re: Shading Plugin + Hadoop Dependency

2016-06-21 Thread Robert Metzger
Hi, This this is how I did it: robert@robert-da ...adoop/flink-shaded-include-yarn-tests (git)-[master] % pwd /home/robert/incubator-flink/flink-shaded-hadoop/flink-shaded-include-yarn-tests robert@robert-da ...adoop/flink-shaded-include-yarn-tests (git)-[master] % mvn clean install [INFO] Scann

Re: Shading Plugin + Hadoop Dependency

2016-06-21 Thread Vijay Srinivasaraghavan
Thanks Rob for trying. In my machine, I don't see any of minikdc related files in the target jar. How did you run the build? RegardsVijay On Tuesday, June 21, 2016 8:39 AM, Robert Metzger wrote: Cool, thank you. After adding the line you've mentioned, my jar contains the missing MiniKd

Re: Shading Plugin + Hadoop Dependency

2016-06-21 Thread Robert Metzger
Cool, thank you. After adding the line you've mentioned, my jar contains the missing MiniKdc class: robert@robert-da ...adoop/flink-shaded-include-yarn-tests (git)-[master] % jar tf target/flink-shaded-include-yarn-tests-1.1-SNAPSHOT.jar | grep minikdc minikdc-krb5.conf minikdc.ldiff org/apache/h

Re: Shading Plugin + Hadoop Dependency

2016-06-21 Thread Vijay Srinivasaraghavan
Hi Rob, You need to include below lines to the pom.xml to resolve the chain dependency error. org.apache.felix maven-bundle-plugin true true RegardsVijay On Tuesday, June 21, 2016 8:27 AM, Robert Metzger wrote: Hi, I didn't even get the shading module to build: [INFO] -

Re: Shading Plugin + Hadoop Dependency

2016-06-21 Thread Robert Metzger
Hi, I didn't even get the shading module to build: [INFO] [INFO] Building flink-shaded-include-yarn-tests 1.1-SNAPSHOT [INFO] [INFO] --

[jira] [Created] (FLINK-4098) Iteration support in Python API

2016-06-21 Thread Geoffrey Mon (JIRA)
Geoffrey Mon created FLINK-4098: --- Summary: Iteration support in Python API Key: FLINK-4098 URL: https://issues.apache.org/jira/browse/FLINK-4098 Project: Flink Issue Type: New Feature

Re: Shading Plugin + Hadoop Dependency

2016-06-21 Thread Vijay Srinivasaraghavan
Hi Rob, Yes I checked but the result jar does not contain the classes from hadoop-minikdc package. RegardsVijay On Tuesday, June 21, 2016 7:59 AM, Robert Metzger wrote: Hi Vijay, did you check if the artifact produced by the "flink-shaded-include-yarn-tests" module contains the requir

Re: Shading Plugin + Hadoop Dependency

2016-06-21 Thread Robert Metzger
Hi Vijay, did you check if the artifact produced by the "flink-shaded-include-yarn-tests" module contains the required classes? Regards, Robert On Mon, Jun 20, 2016 at 6:18 PM, Vijay Srinivasaraghavan < vijikar...@yahoo.com.invalid> wrote: > Hello, > I was trying to include "hadoop-minikdc" co

Re: kafka partition assignment

2016-06-21 Thread Michal Hariš
Hi Tai, OK, thanks for confirming. I understand that streaming shuffle is cheaper than batch-spill shuffle, but nevertheless may be unacceptable in large volume applications - it is still a network shuffle and that's the biggest part of the cost. Now on the point of the trade off between load-bala

Re: Advice - Drools in Flink

2016-06-21 Thread Robert Metzger
Hi Anton, I think embedding Drools into Flink should be doable. Drools seems to be implemented in Java, so you can probably call the engine from Flink. I would start putting a flatMap() operator into a stream. In the operator, I would start the Drools engine (probably in the open() method) and the

[jira] [Created] (FLINK-4097) Cassandra Sink throws NPE on closing if server is not available

2016-06-21 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-4097: - Summary: Cassandra Sink throws NPE on closing if server is not available Key: FLINK-4097 URL: https://issues.apache.org/jira/browse/FLINK-4097 Project: Flink