Re: [Flink-9407] Question about proposed ORC Sink !

Yes, it should be exit. Thanks to Ted Yu. Very exactly! Cheers Zhangminglei > 在 2018年6月23日，下午12:40，Ted Yu 写道： > > For #1, the word exist should be exit, right ? > Thanks > > Original message > From: zhangminglei <18717838...@163.com> > Date: 6/23/18 10:12 AM (GMT+08:00) > To

Re: [Flink-9407] Question about proposed ORC Sink !

For #1, the word exist should be exit, right ?Thanks Original message From: zhangminglei <18717838...@163.com> Date: 6/23/18 10:12 AM (GMT+08:00) To: sagar loke Cc: dev , user Subject: Re: [Flink-9407] Question about proposed ORC Sink ! Hi, Sagar. 1. It solves the issue par

Re: [Flink-9407] Question about proposed ORC Sink !

Hi, Sagar. > 1. It solves the issue partially meaning files which have finished > checkpointing don't show .pending status but the files which were in progress > when the program exists are still in .pending state. Ans: Yea, Make the program exists and in that time if a checkpoint does no

Re: Flink 1.5 Yarn Connection unexpectedly closed

I don't know why yet, but I did figure it out. After my sample long running map reduce test ran fine all night I tried a ton of things. Turns out there is a difference between env.execute() and env.collect(). My flow had reading from HDFS, decrypting, processing, and finally writing to HDFS, at

String Interning

I have a job where I read data from Kafka, do some processing on it, and write it to a database. When I read data out of Kafka, I put it into an object that has a String field based on the Kafka message key. The possible values for the message key are tightly constrained so there are fewer than

RE: Remote Debugging Flink

In conf.yaml file, replace the setting ‘env.java.opts: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005" ‘ as mentioned in the link with the below setting env.java.opts.taskmanager: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005" Regards, Teena From: Th

RE: Debug job execution from savepoint

Great ! Thanks for your help. I think I will have all the functionalities I need for my use case with these test classes. Regards, Manuel De : Chesnay Schepler Envoyé : vendredi 22 juin 2018 15:03:01 À : Haddadi Manuel Cc : user@flink.apache.org Objet : Re: De

Re: Debug job execution from savepoint

Similarly, the AbstractOperatorRestoreTestBase shows how to start en embedded cluster and submit a job from a savepoint. see * #restoreJob * #createJobGraph * MiniClusterResource On 22.06.2018 13:53, Tzu-Li (Gordon) Tai wrote: Hi, The tests in Flink uses a `AbstractStreamOperatorTestHarness` t

Re: Memory Leak in ProcessingTimeSessionWindow

Stefan, All,  If there are no further thoughts on this I am going to switch my app to low level Process API. I still think there is an easier solution here which I am missing but I will revisit that after I fix Production issue. Thanks, Ashish Sent from Yahoo Mail for iPhone On Thursday, June

Re: Remote Debugging Flink

Hi, did you specify the `suspend=y` flag? This could stop the TaskManagers from connecting to the ResourceManager and offering their slots. This can then ultimately lead to the NoResourceAvailableException if you don't allow the TaskManagers to continue. So my suspicion would be that you don't con

RE: Debug job execution from savepoint

Hi, The tests in Flink uses a `AbstractStreamOperatorTestHarness` that allows wrapping an operator, input elements into the operator, getting the outputs, and also snapshotting / restoring operator state. I’m not sure of your specific case, but in general that test harness utility can be used t

Remote Debugging Flink

*Hi, I am new to flink and trying to remote debug flink cluster. I followed the instructions mentioned in the link[1] but when I do that I could not run the SocketWindowWordCount example(any example at all). The main problem is there are no task slots or task manager initiated. Then I followed the

[NOTICE] Flink Kinesis Producer for versions 1.4.2 and below needs to be built with higher AWS KPL version

Hi, This is a notice for users who use the Flink Kinesis connector to produce data to AWS Kinesis Streams, with Flink versions 1.4.2 or below. For Flink versions 1.4.2 and below, the KPL client version used by default in the Kinesis connectors, KPL 0.12.5, is no longer supported by AWS Kinesis

Re: Flink 1.5 Yarn Connection unexpectedly closed

Hi Garrett, have you set a restart strategy for your job [1]? In order to recover from failures you need to specify one. Otherwise Flink will terminally fail the job in case of a failure. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/restart_strategies.html Cheers, Till On

RE: Debug job execution from savepoint

Hi Fabian, thanks for your response. Here is the Jira issue : https://issues.apache.org/jira/browse/FLINK-9644 Would you have any devlopper tips on how testing my intializeState methods to validate that my savepoints are corrects ? Thank you again. Regards, Manuel ___

Re: Strictly use TLSv1.2

Great, thank you! 2018-06-22 10:16 GMT+02:00 Vinay Patil : > Hi Fabian, > > Created a JIRA ticket : https://issues.apache.org/jira/browse/FLINK-9643 > > Regards, > Vinay Patil > > > On Fri, Jun 22, 2018 at 1:25 PM Fabian Hueske wrote: > >> Hi Vinay, >> >> This looks like a bug. >> Would you mind

Re: Strictly use TLSv1.2

Hi Fabian, Created a JIRA ticket : https://issues.apache.org/jira/browse/FLINK-9643 Regards, Vinay Patil On Fri, Jun 22, 2018 at 1:25 PM Fabian Hueske wrote: > Hi Vinay, > > This looks like a bug. > Would you mind creating a Jira ticket [1] for this issue? > > Thank you very much, > Fabian >

Re: Strictly use TLSv1.2

Hi Vinay, This looks like a bug. Would you mind creating a Jira ticket [1] for this issue? Thank you very much, Fabian [1] https://issues.apache.org/jira/projects/FLINK 2018-06-21 9:25 GMT+02:00 Vinay Patil : > Hi, > > I have deployed Flink 1.3.2 and enabled SSL settings. From the ssl debug >

Re: SQL Do Not Support Custom Trigger

Hi, Although this solution looks straight-forward, custom triggers cannot be added that easily. The problem is that a window operator with a Trigger that emit early results produces updates, i.e., results that have been emitted might be updated later. The default Trigger only emits the final resu