Re: Monitoring single-run job statistics

Good luck with the Thesis! On Wed, Jan 6, 2016 at 6:18 PM, Filip Łęczycki wrote: > Hi Stephan, > > Thank you for you answer. I would love to contribute but currently I have > no capacity as I am buried with my thesis. > > I will reach out after graduating :) > > Bestr regards. > Filip > > Pozdra

Re: flink kafka scala error

I did the solve problem by changing the scala version for Kafka library as i download the scala_2.11 version of flink ( flink-0.10.1-bin-hadoop27-scala_2.11.tg z). Before: org.apache

Re: sources not available for flink-streaming

I also saw that this is not working for SNAPSHOT releases (for stable releases it seems to work). We are publishing the source artifacts, as you can see here https://repository.apache.org/content/repositories/snapshots/org/apache/flink/flink-streaming-scala/1.0-SNAPSHOT/ Maybe its an issue with

sources not available for flink-streaming

*IntelliJ reports the following:* *Cannot download sources* Sources not found for: org.apache.flink:flink-streaming-scala:1.0-SNAPSHOT Would it be possible to publish sources for this artifact? -- *Alex Rovner* *Director, Data Engineering * *o:* 646.759.0052 * *

Re: Monitoring single-run job statistics

Hi Stephan, Thank you for you answer. I would love to contribute but currently I have no capacity as I am buried with my thesis. I will reach out after graduating :) Bestr regards. Filip Pozdrawiam, Filip Łęczycki 2016-01-05 10:35 GMT+01:00 Stephan Ewen : > Hi Filip! > > There are thoughts an

Re: Problem to show logs in task managers

Maybe the isConverged() method is never called? For making that sure, just throw a RuntimeException inside the method to see whats happening. On Wed, Jan 6, 2016 at 3:58 PM, Ana M. Martinez wrote: > Hi Till, > > I am afraid it does not work in any case. > > I am following the steps you indicate

Re: Problem to show logs in task managers

Hi Till, I am afraid it does not work in any case. I am following the steps you indicate on your websites (for yarn configuration and loggers with slf4j): 1) Enable log aggregation in yarn-site: https://ci.apache.org/projects/flink/flink-docs-release-0.7/yarn_setup.html#log-files 2) Include Lo

Re: Unit testing support for flink application?

Hi Filipe, The problem your encountering most likely stems from the fact that Flink serializes all operators before running them in the (local) cluster. During this process all outside references inside your sink are lost. In the thread you've found are two solutions for this: Use the collect sin

Re: kafka integration issue

Wow, okay, you must have hit exactly the point in time when the update was pushed ;-) On Wed, Jan 6, 2016 at 2:18 PM, Alex Rovner wrote: > Updating to latest version worked. Thanks! My git repo was less than 1 day > old :-( > > On Wed, Jan 6, 2016 at 4:54 AM Stephan Ewen wrote: > >> "java.lang.

Re: kafka integration issue

Updating to latest version worked. Thanks! My git repo was less than 1 day old :-( On Wed, Jan 6, 2016 at 4:54 AM Stephan Ewen wrote: > "java.lang.NoSuchMethodError" in Java virtually always means that the > code is compiled against a different version than executed. > > The version in "~/git/fl

Re: Flink on EMR Question

Would it cause problems if I remove it from the "flink-runtime" pom? Seems strange to have a dependency there that we do not even use... On Wed, Jan 6, 2016 at 12:07 PM, Ufuk Celebi wrote: > @Stephan: It was added to the dependency management section in order to > enforce a higher version for S

Re: Local collection data sink for the streaming API

Perfect, thanks! Filipe On Tue, Jan 5, 2016 at 6:23 PM, Gábor Gévay wrote: > Try the getJavaStream method of the scala DataStream. > > Best, > Gábor > > > > > 2016-01-05 19:14 GMT+01:00 Filipe Correia : >> Hi Gábor, Thanks! >> >> I'm using Scala though. DataStreamUtils.collect() depends on >> or

Re: Flink on EMR Question

@Stephan: It was added to the dependency management section in order to enforce a higher version for S3 client, because it was causing problems earlier. > On 06 Jan 2016, at 11:14, Chiwan Park wrote: > > Great! Thanks for addressing! > >> On Jan 6, 2016, at 5:51 PM, Stephan Ewen wrote: >> >>

Re: Flink on EMR Question

Great! Thanks for addressing! > On Jan 6, 2016, at 5:51 PM, Stephan Ewen wrote: > > At a first look, I think that "flink-runtime" does not need Apache Httpclient > at all. I'll try to simply remove that dependency... > > On Wed, Jan 6, 2016 at 7:14 AM, Chiwan Park wrote: > Hi, > > Thanks for

Re: kafka integration issue

"java.lang.NoSuchMethodError" in Java virtually always means that the code is compiled against a different version than executed. The version in "~/git/flink/" must be slightly outdated. Can you pull the latest update of the 1.0-SNAPSHOT master and rebuild the code? Stephan On Tue, Jan 5, 2016 a

Re: flink kafka scala error

Hi Madhukar, could you check whether your Flink installation contains the flink-dist-0.10.1.jar in the lib folder? This file contains the necessary scala-library.jar which you are missing. You can also remove the line org.scala-lang:scala-library which excludes the scala-library dependency to be i

Re: Flink on EMR Question

At a first look, I think that "flink-runtime" does not need Apache Httpclient at all. I'll try to simply remove that dependency... On Wed, Jan 6, 2016 at 7:14 AM, Chiwan Park wrote: > Hi, > > Thanks for answering me! > > It is happy to hear the problem will be addressed. :) > > About question 2,