[jira] [Created] (FLINK-3149) mvn test fails on flink-runtime because curator classes not found

2015-12-08 Thread jun aoki (JIRA)
jun aoki created FLINK-3149: --- Summary: mvn test fails on flink-runtime because curator classes not found Key: FLINK-3149 URL: https://issues.apache.org/jira/browse/FLINK-3149 Project: Flink Issue

Re: Task Parallelism in a Cluster

2015-12-08 Thread Stephan Ewen
Hi Ali! In the case you have, the sequence of source-map-filter ... forms a pipeline. You mentioned that you set the parallelism to 16, so there should be 16 pipelines. These pipelines should be completely independent. Looking at the way the scheduler is implemented, independent pipelines should

flink-dist packaging including unshaded classes

2015-12-08 Thread Nick Dimiduk
Hi there, I'm attempting to build locally a flink based on release-0.10.0 + FLINK-3147. When I build from this sandbox, the resulting flink-dist.jar contains both shanded and unshaded jars. In my case, this results in a runtime conflict in my application, where com.google.common.base.Stopwatch fro

[jira] [Created] (FLINK-3148) Support configured serializers for shipping UDFs

2015-12-08 Thread Nick Dimiduk (JIRA)
Nick Dimiduk created FLINK-3148: --- Summary: Support configured serializers for shipping UDFs Key: FLINK-3148 URL: https://issues.apache.org/jira/browse/FLINK-3148 Project: Flink Issue Type: Impr

[jira] [Created] (FLINK-3147) HadoopOutputFormatBase should expose CLOSE_MUTEX for subuclasses

2015-12-08 Thread Nick Dimiduk (JIRA)
Nick Dimiduk created FLINK-3147: --- Summary: HadoopOutputFormatBase should expose CLOSE_MUTEX for subuclasses Key: FLINK-3147 URL: https://issues.apache.org/jira/browse/FLINK-3147 Project: Flink

[jira] [Created] (FLINK-3146) Expose the Kafka consumer lag as a metric in the web interface for Kafka 0.8.x

2015-12-08 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3146: - Summary: Expose the Kafka consumer lag as a metric in the web interface for Kafka 0.8.x Key: FLINK-3146 URL: https://issues.apache.org/jira/browse/FLINK-3146 Projec

Re: Migrating Spark to Flink

2015-12-08 Thread Robert Metzger
Is this a duplicate with your Stackoverflow question? http://stackoverflow.com/questions/34146800/convert-reducebykey-from-spark-to-flink On Tue, Dec 8, 2015 at 2:25 PM, Humberto Moura < humbe...@humbertomoura.com.br> wrote: > Hello, guys > > > I'm migrating a Spark code to Flink, and I realized

[jira] [Created] (FLINK-3145) Storm examples can't be run without flink-java as dependency

2015-12-08 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-3145: - Summary: Storm examples can't be run without flink-java as dependency Key: FLINK-3145 URL: https://issues.apache.org/jira/browse/FLINK-3145 Project: Flink

Migrating Spark to Flink

2015-12-08 Thread Humberto Moura
Hello, guys I'm migrating a Spark code to Flink, and I realized that Spark reduceByKey doesn't exist. |the snip code I'm struggle: reduceByKey((x,y)=>(x._1 +y._1,((x._2)++y._2))) So, with a help of a friend I've tried convert: ||groupBy(0).reduce {(v1,v2)=>(v1._1 +v2._1,((v1._2)++v2._2))} bu

[jira] [Created] (FLINK-3144) [storm] LocalCluster prints nothing without a configured logger

2015-12-08 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-3144: - Summary: [storm] LocalCluster prints nothing without a configured logger Key: FLINK-3144 URL: https://issues.apache.org/jira/browse/FLINK-3144 Project: Flin

[jira] [Created] (FLINK-3143) Update Clojure Cleaner's ASM references to ASM5

2015-12-08 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-3143: - Summary: Update Clojure Cleaner's ASM references to ASM5 Key: FLINK-3143 URL: https://issues.apache.org/jira/browse/FLINK-3143 Project: Flink Issue

Re: The null in Flink

2015-12-08 Thread Aljoscha Krettek
Yes, I agree, these are the steps to take. Thanks for creating the issues. > On 08 Dec 2015, at 07:18, Li, Chengxiang wrote: > > Chengxiang