Re: Unable to write snapshots to S3 on EMR

Hi Fabian, 1) I have looked at the linked docs, and from what I can tell no setup should really need to be done to get Flink working(Other than downloading the correct binaries, which I believe I did) 2) I have downloaded the Flink 1.3.2 binaries(flink-1.3.2-bin- hadoop27-scala_2.11.tgz

Re: Unable to write snapshots to S3 on EMR

Hi Andy, I'm not an AWS expert, so I'll just check on some common issues. I guess you already had a look at the Flink docs for AWS/EMR but I'll post the link just be to sure [1]. Since you are using Flink 1.3.2 (EMR 5.8.0 comes with Flink 1.3.1) did you built Flink yourself or did you download t

Re: Unable to write snapshots to S3 on EMR

Hi Fabian, Sorry, I just realized I forgot to include that part. The error returned is: java.lang.NoSuchMethodError: org.apache.hadoop.conf.Configuration.addResource(Lorg/apache/hadoop/conf/Configuration;)V at com.amazon.ws.emr.hadoop.fs.EmrFileSystem.initialize(EmrFileSystem.java:93) a

Re: Unable to write snapshots to S3 on EMR

Hi Andy, can you describe in more detail what exactly isn't working? Do you see error messages in the log files or on the console? Thanks, Fabian 2017-10-02 15:52 GMT+02:00 Andy M. : > Hello, > > I am about to deploy my first Flink projects to production, but I am > running into a very big hur

[jira] [Created] (FLINK-7755) Null values are not correctly handled by batch inner and outer joins

Fabian Hueske created FLINK-7755: Summary: Null values are not correctly handled by batch inner and outer joins Key: FLINK-7755 URL: https://issues.apache.org/jira/browse/FLINK-7755 Project: Flink

[jira] [Created] (FLINK-7754) Complete termination future after actor has been stopped.

Till Rohrmann created FLINK-7754: Summary: Complete termination future after actor has been stopped. Key: FLINK-7754 URL: https://issues.apache.org/jira/browse/FLINK-7754 Project: Flink Issue

[jira] [Created] (FLINK-7753) HandlerUtils should close the channel on error responses

Eron Wright created FLINK-7753: --- Summary: HandlerUtils should close the channel on error responses Key: FLINK-7753 URL: https://issues.apache.org/jira/browse/FLINK-7753 Project: Flink Issue Ty

[jira] [Created] (FLINK-7752) RedirectHandler should execute on the IO thread

Eron Wright created FLINK-7752: --- Summary: RedirectHandler should execute on the IO thread Key: FLINK-7752 URL: https://issues.apache.org/jira/browse/FLINK-7752 Project: Flink Issue Type: Sub-t

Unable to write snapshots to S3 on EMR

Hello, I am about to deploy my first Flink projects to production, but I am running into a very big hurdle. I am unable to launch my project so it can write to an S3 bucket. My project is running on an EMR cluster, where I have installed Flink 1.3.2. I am using Yarn to launch the application,

[jira] [Created] (FLINK-7751) Documentation for Homebrew installation is unclear

Chris Ward created FLINK-7751: - Summary: Documentation for Homebrew installation is unclear Key: FLINK-7751 URL: https://issues.apache.org/jira/browse/FLINK-7751 Project: Flink Issue Type: Improv

Re: Logistic Regression in Flink ML

Hi Bowen, currently, we are quite short on people working on the ML part. You could help us with the reviewing process if you tried it out. Do you think this would be possible? Cheers, Till On Mon, Oct 2, 2017 at 2:39 AM, Bowen Li wrote: > Hi guys, > > I checked that FlinkML still doesn't have

[jira] [Created] (FLINK-7750) Strange behaviour in savepoints

Razvan created FLINK-7750: - Summary: Strange behaviour in savepoints Key: FLINK-7750 URL: https://issues.apache.org/jira/browse/FLINK-7750 Project: Flink Issue Type: Bug Components: Core

[jira] [Created] (FLINK-7749) remove ResultPartitionWriter

Nico Kruber created FLINK-7749: -- Summary: remove ResultPartitionWriter Key: FLINK-7749 URL: https://issues.apache.org/jira/browse/FLINK-7749 Project: Flink Issue Type: Sub-task Compone

[jira] [Created] (FLINK-7748) remove event listener behaviour from ResultPartitionWriter

Nico Kruber created FLINK-7748: -- Summary: remove event listener behaviour from ResultPartitionWriter Key: FLINK-7748 URL: https://issues.apache.org/jira/browse/FLINK-7748 Project: Flink Issue Ty

Re: partial upgrade

Hi Chen, I think in a long-term perspective it makes sense to support things like this. The next big step is dynamic scaling without stopping the execution. Partial upgrades could be addressed afterwards, but I'm not aware of any plans. Until then, I would recommend a different architecture

[jira] [Created] (FLINK-7747) remove RecordWriter wrapper

Nico Kruber created FLINK-7747: -- Summary: remove RecordWriter wrapper Key: FLINK-7747 URL: https://issues.apache.org/jira/browse/FLINK-7747 Project: Flink Issue Type: Improvement Compo

[jira] [Created] (FLINK-7746) move ResultPartitionWriter#writeBufferToAllChannels up into ResultPartition

Nico Kruber created FLINK-7746: -- Summary: move ResultPartitionWriter#writeBufferToAllChannels up into ResultPartition Key: FLINK-7746 URL: https://issues.apache.org/jira/browse/FLINK-7746 Project: Flink

[jira] [Created] (FLINK-7745) add tests for ensuring NetworkBufferPool overprovisioning behaviour

Nico Kruber created FLINK-7745: -- Summary: add tests for ensuring NetworkBufferPool overprovisioning behaviour Key: FLINK-7745 URL: https://issues.apache.org/jira/browse/FLINK-7745 Project: Flink

Re: Metrics to data connectors

@Chesnay do you know more about it? Am 10/1/17 um 4:45 PM schrieb Michael Fong: Hi, all, Is there any existing metrics implemented to any src/sink connectors? I am thinking to add some metrics to C* connector that might help to give a early sign of scalability problem (like back pressure) fro