Re: Problem with passing arguments to Flink Web Submission Client

2015-12-21 Thread Matthias J. Sax
Thanks for opening a JIRA. I used "info" which yield the error (forgot it in the mail): > bin/flink info myJarFile.jar -f flink -i -m 1 -Matthias On 12/21/2015 10:36 AM, Filip Łęczycki wrote: > Hi, > > Regarding the CLI, I have been using >>bin/flink run myJarFile.jar -f flink -i -m 1 > an

How do failovers work on yarn?

2015-12-21 Thread Niels Basjes
Hi, When I start a yarn-session I found that a file is written with a hostname+port number in a config file on my machine. Apparently this is the place where the job manager van be found. Question: What happens if that node on the cluster goes down? I expect that Yarn will reallocate the job mana

Re: Configure log4j with XML files

2015-12-21 Thread Robert Metzger
as an additional note: Flink is sending all files in the /lib folder to all YARN containers. So you could place the XML file in "/lib" and override the properties. I think you need to delete the log4j properties from the conf/ directory, then at least on YARN, we'll not set the -Dlog4j.configurati

Re: Configure log4j with XML files

2015-12-21 Thread Till Rohrmann
Hi Gwenhaël, as far as I know, there is no direct way to do so. You can either adapt the flink-daemon.sh script in line 68 to use a different configuration or you can test whether the dynamic property -Dlog4j.configurationFile:CONFIG_FILE overrides the -Dlog4j.confguration property. You can set th

Configure log4j with XML files

2015-12-21 Thread Gwenhael Pasquiers
Hi everybody, Could it be possible to have a way to configure log4j with xml files ? I've looked into the code and it looks like the properties files names are hardcoded. However we have the need to use xml : - We log everything into ELK (Elasticsearch / Logstash / Kibana) using Socke

Re: Specify jobmanager port in HA mode

2015-12-21 Thread Ufuk Celebi
> On 17 Dec 2015, at 19:36, Cory Monty wrote: > > Hey Ufuk, > > We can try building it, but it might be a week or so given the holidays. > We're currently targeting development against 1.0.0-SNAPSHOT and that release > is OK unless 1.0 release is more than 2-3 months away. OK, that’s the exp

Re: Serialisation problem

2015-12-21 Thread Fabian Hueske
Hi, In your program, you apply a distinct transformation on a data set that has a (nested) GValue[] type. Distinct requires that all fields are comparable with each other. Therefore all fields of the data sets' type must be valid key types. However, Flink does not support object arrays as keys typ

Re: Problem with passing arguments to Flink Web Submission Client

2015-12-21 Thread Filip Łęczycki
Hi, Regarding the CLI, I have been using >bin/flink run myJarFile.jar -f flink -i -m 1 and it is working perfectly fine. Is there a difference between this two ways of submitting a job ("bin/flink MyJar.jar" and "bin/flink run MyJar.jar")? I will open a Jira. Best Regards, Filip Łęczycki Pozdr