Re: Flink and AWS S3 integration: java.lang.NullPointerException: null uri host

i solved it by myself, with the help of some debugging. i used s3:///mybucket/ but it needs to be s3://mybucket/some_folder ... 2 slashes, and also a folder needs to be specified ... -- *Fabian WollertZalando SE* E-Mail: fabian.woll...@zalando.de Location: ZMAP

RE: Looking for a working POM file example for EMR cluster

Hi, Did you check solutions from here - https://stackoverflow.com/questions/48904881/could-not-resolve-substitution-to-a-value-akka-stream-materializer-in-aws-la Regards, Georgi Stoyanov From: Sandybayev, Turar (CAI - Atlanta) Sent: Thursday, May 31, 2

How to set UserGroupInformation?

Hi all I'm trying to write data to HDFS in directory "/home/hbase/XXX". Only "hbase" user can write to the directory. The problem is: I submit a job to yarn. The job will be a "yarn" user to write data, while "yarn" user is not allowed to write to "/home/hbase/XXX". Is there any method that I can

Re: Trigerring Savepoint for the Flink Job

Thanks for the reply Rong. We had updated Athenax to version 1.4. I had checked Flink 1.4, it's rest endpoint dose not support only creating Savepoint. It has cancel With Savepoint. I think creating Savepoint is supported in 1.5. Since we can't upgrade to 1.5 at the moment it would like to find a

Re: Trigerring Savepoint for the Flink Job

Hi Anil, I am actually one of the engineers maintaining Uber's AthenaX open-source platform. As for now, AthenaX is still running on Flink 1.3.2 so it might be weird to follow the 1.4.2 release document. (We are working on upgrading to latest 1.5 release) For your question regarding savepoints, A

REMINDER: Apache EU Roadshow 2018 in Berlin is less than 2 weeks away!

Hello Apache Supporters and Enthusiasts This is a reminder that our Apache EU Roadshow in Berlin is less than two weeks away and we need your help to spread the word. Please let your work colleagues, friends and anyone interested in any attending know about our Apache EU Roadshow event. We h

Looking for a working POM file example for EMR cluster

Hi, I'm looking for a sample POM file that works when running on EMR cluster. I'm new to Flink and EMR, so I'm simply following AWS EMR documentation on Flink and I am creating a Step and submitting my program JAR file. My program is just a slight modification of the Wikipedia example. I was

Trigerring Savepoint for the Flink Job

I am using Flink 1.4.2. I have forker Uber's AthenaX project . The Flink jobs are deployed in Yarn cluster. I needed to save the Savepoint for all the jobs everyday. ClusterClient

RE: Use element of the DataStream in parameter of RichMapFunction (open function not called)

Hello Fabian, Thanks for this quick answer ! I understand I cannot mix the custom map function with the lambda definition. The workaround we found for the issue was to pass the function to get the key as a parameter to the CustomMapFunction instead of the key itself, and now it works ! Regards

Re: Flink 1.2.1 - Job initial start time

Adding my team mate. ~ Abhinav Bajaj From: "Bajaj, Abhinav" Date: Thursday, May 31, 2018 at 9:30 AM To: "user@flink.apache.org" Subject: Flink 1.2.1 - Job initial start time Hi, Is there a way to know the initial start time of a job? I am looking for the first time job was started and not th

Flink 1.2.1 - Job initial start time

Hi, Is there a way to know the initial start time of a job? I am looking for the first time job was started and not the time of the restart. It seems the “start-time” reported by monitoring API under joboverview

File does not exist prevent from Job manager to start .

Hi, Im having some wierd issue with the JM recovery , I using HDFS and ZOOKEEPER for HA stand alone cluster . Iv stop the cluster change some parameters in the flink conf (Memory). But now when i start the cluster again im having an error that preventing from JM to start. somehow the checkpoint

Re: is it OK to PR just for code comment errors?

Yes, Chesnay was right if the error code comment doesn't affect the clarity of the message. Sorry for the confusion. -- Rong On Thu, May 31, 2018 at 7:54 AM, Chesnay Schepler wrote: > For typos in the documentation or javadocs feel free to open a PR. > > I would leave code comments alone as lon

Re: is it OK to PR just for code comment errors?

I think this is classified as "hotifx" in the contributor list, no JIRA should be needed. *" Typo fixes that have no associated JIRA issue should be named following this pattern: `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGene

Re: is it OK to PR just for code comment errors?

For typos in the documentation or javadocs feel free to open a PR. I would leave code comments alone as long as the typo doesn't affect the clarity/semantics of the comment, to not fragment the git history unnecessarily. On 31.05.2018 11:40, makeyang wrote: I am not sure if it is OK to PR jus

Re: is it OK to PR just for code comment errors?

Hi Makeyang, It is definitely OK to open a PR for correcting code comment errors. Especially for the ones that are misleading or provide incorrect information to users. I am not sure if it is required if only for a one-liner change, but please file a JIRA ticket for the code comment error as well

Re: Ask about convert KafkaTableSource to table

Please take a look at the table api/SQL doc [1] below on how to register a table/tablesource. Thanks. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/table/common.html#register-a-table On Thu, May 31, 2018 at 6:22 AM, Radhya Sahal wrote: > Hi, > > Could you help me which is

Ask about convert KafkaTableSource to table

Hi, Could you help me which is the integrated versions of Java/Kafka/ Flink which can work without errors. I have a problem when I read streamed data from KafkaTableSource to table. I use this command tableEnv.registerDataStream("sensors", myKafkaTableSource) But, there it doesn't run (i.

Re: Multiple Task Slots support in Flink 1.5

Hi, We're using multiple slots per TaskManager with legacy mode, and everything works fine. For the new default mode, it also seems to works for us, so I'm not sure what is not supported. May be someone from Flink team could clarify. Best regards, Kien On 5/31/2018 4:26 AM, Abdul Qadeer

some default config questions

why jobmanager.execution.failover-strategy default value is full not Individual? why taskmanager.jvm-exit-on-oom default value is false not true? code is flink 1.5 -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

is it OK to PR just for code comment errors?

I am not sure if it is OK to PR just for code comment errors since I found there is one when I am reading code -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Use element of the DataStream in parameter of RichMapFunction (open function not called)

Hi Isabelle, Welcome to the Flink user mailing list! You are mixing up the two ways to specify a function: 1. Defining a function as a class / object and passing an instance in the map() method. Given your CustomMapFunction class, this looks as follows: stream.keyBy(...).map(new CustomMapFunctio

Re: Flink and AWS S3 integration: java.lang.NullPointerException: null uri host

I'm running it in docker on EC2, cant use EMR ... yes i followed those instructions. Cheers -- *Fabian WollertZalando SE* E-Mail: fabian.woll...@zalando.de Am Do., 31. Mai 2018 um 03:07 Uhr schrieb Bowen Li : > Did you run Flink on AWS EMR or somewhere else? Have you read and followed > ins

Re: NPE in flink sql over-window

Hi Yan, Thanks for the details and for digging into the issue. If I got it right, the NPE caused the job failure and recovery (instead of being the result of a recovery), correct? Best, Fabian 2018-05-31 7:00 GMT+02:00 Yan Zhou [FDS Science] : > Thanks for the replay. > > > Yes, it only happen