Re: External checkpoint metadata in Flink 1.5.x

Hi, To be fair my "workaround" is to restore the previous behaviour in our Flink build as it only takes a few lines of code :) To explain why this is convenient I can describe how we use the checkpoints/savepoints. For every Streaming Application (not flink job) we have a unique name from which t

Re: Flink job hangs/deadlocks (possibly related to out of memory)

Hi Zhijiang, The problem is that no other task failed first. We have a task that sometimes just stops processing data, and when we cancel it, we see the logs messages saying: " Task (...) did not react to cancelling signal for 30 seconds, but is stuck in method: org.apache.flink.core.memory.Data

Re: Taskmanager SSL fails looking for Subject Alternative IP Address

Thanks for reporting this. Given that hostname verification seems to be the issue, I would assume that the TaskManager somehow advertises a hostname in a form that is incompatile with the verification in some setups. While it would be interesting to dig deeper into why this happens, I think we ne

Flink CLI properties with HA

Hello When HA is enabled in the flink cluster and if I've to submit job via flink CLI then in the flink-conf.yaml of flink CLI should contain this properties - high-availability: zookeeper high-availability.cluster-id: flink high-availability.zookeeper.path.root: flink high-availability.storageDir

[ANNOUNCE] Apache Flink 1.5.1 released

The Apache Flink community is very happy to announce the release of Apache Flink 1.5.1, which is the first bugfix release for the Apache Flink 1.5 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming

Re: Need assistance : creating remote environment

Hi, what is flip6? "Flink Improvement Proposal" ? How to enable/disable flip mode in below cases, 1. While starting cluster using ./start-cluster.sh 2. While connecting using ExecutionEnvironment.createRemoteEnvironment("localhost", 6123, "xyz.jar") Thank you. On Thu, Jul 12, 2018 at 9:32 PM

Cannot configure akka.ask.timeout

Hello, I have problems setting configuration parameters for Akka in Flink 1.5.0. When I run a job I get the exception listed below which states that Akka timed out after 1ms. I tried to increase the timeout by following the Flink configuration documentation. Specifically I did the following

Re: Need assistance : creating remote environment

FLIP-6 is/was an initiative to rework Flink distributed model. Contrary to my previous reply this wasn't introduced in 1.4, please disregard explanation a). Thus, the only remaining explanation is that you're running 1.5 in the client/IDE. I heavily recommend to make sure that your client/I

TimeWindow doesn't trigger reduce function

Hi, My stream data is in a type of Tuple2 that contains the timestamp (in second) and data, respectively. The source will generate 120 sample every second. Using the following code I want to get data in every second and then apply the reduce function on them. temp.keyBy( 0).timeWindow(Time.seconds

回复：Flink job hangs/deadlocks (possibly related to out of memory)

Hi Gerard, I thought the failed task triggers cancel process before, now I am clear that you cancel the task when it stops processing data. I think you can jstack the process to find where task thread is blocked instead of canceling it, then we may find some hints. In addition, the following st

Re: [ANNOUNCE] Apache Flink 1.5.1 released

Great to hear. Big thank you to the community for the hard work and to Chesnay for being our release manager. Cheers, Till On Fri, Jul 13, 2018 at 12:05 PM Chesnay Schepler wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink 1.5.1, which is the first bugf

Re: [ANNOUNCE] Apache Flink 1.5.1 released

Thanks Chesnay, great job! Thanks, Vino 2018-07-13 20:20 GMT+08:00 Till Rohrmann : > Great to hear. Big thank you to the community for the hard work and to > Chesnay for being our release manager. > > Cheers, > Till > > On Fri, Jul 13, 2018 at 12:05 PM Chesnay Schepler > wrote: > >> The Apache

Re: TimeWindow doesn't trigger reduce function

Hi Soheil, It seems you job stops within 1 second? The processing time window doesn't output data if time hasn't reach the window end. While event time window will output a final watermark during close() to avoid this problem. You can try to increase the running time of your job to get the output

Re: [ANNOUNCE] Apache Flink 1.5.1 released

Cool, thanks to Chesnay! Best, Hequn On Fri, Jul 13, 2018 at 8:25 PM, vino yang wrote: > Thanks Chesnay, great job! > > Thanks, > Vino > > 2018-07-13 20:20 GMT+08:00 Till Rohrmann : > >> Great to hear. Big thank you to the community for the hard work and to >> Chesnay for being our release mana

Re: [ANNOUNCE] Apache Flink 1.5.1 released

Good job everyone and Chesnay for being the release manager! On 13/07/18 14:34, Hequn Cheng wrote: > Cool, thanks to Chesnay! > > Best, Hequn > > On Fri, Jul 13, 2018 at 8:25 PM, vino yang > wrote: > > Thanks Chesnay, great job! > > Thanks, > Vino > >

Flink on Mesos: containers question

Can someone please clarify how Flink on Mesos in containerized? On 5-node Mesos cluster I started Flink (1.4.2) with two Task Managers. Mesos shows "flink" task and two "taskmanager" tasks, all on the same VM. On that VM I see one Docker container running a process that seems to be Mesos App Mas

Re: 答复: 答复: TumblingProcessingTimeWindow emits extra results for a same window

Hi Youjun, The rowtime value in udf:EXTRACT(EPOCH FROM rowtime) is different from the rowtime value of window. Sql will be parsed and translated into some nodes, Source -> Calc -> Window -> Sink. The Calc is the input node of Window and the udf is part of Calc instead of Window. So the max_ts and

Using an Akka Actor System with Flink

I have a few libraries for doing IO, with large amounts of data, that use akka-http. Now. I have a Flink application, where I need to use those libraries. How do I use a common actor-system across my flink app? I mean, AFAIU, flink apps will be distributed and the same pipeline could end up creati

akka.stream.materializer exception

Hi, I have a jar file that subscribes streaming data from Kafka and then executes some Flink queries. When the program runs locally, it works fine. But, when I run the jar file, I got these exceptions which sound that akka.stream.materializer causes the problem, I added akka.stream dependencies

Re: Need assistance : creating remote environment

Hi Chesnay, Now server and client/IDE both are using 1.4.0. When I submit job from IDE getting below exception, com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka.remote.artery' I have used shadow jar. Thought problem is with merging jars. But I could not fi

SinkFunction invoke method signature

Dear Flink Users, I noticed my sink's `invoke` method was deprecated, so I went to update it, I'm a little surprised by the new method signature, especially on Context (copy+pasted below for ease of discussion). Shouldn't Context be Context not Context ? based on the docs? I'm having a hard ti

Re: SinkFunction invoke method signature

+1 We encountered the exact same issue today. On Fri, Jul 13, 2018 at 10:51 PM Philip Doctor wrote: > Dear Flink Users, > I noticed my sink's `invoke` method was deprecated, so I went to update > it, I'm a little surprised by the new method signature, especially on > Context (copy+pasted below

Re: Loading Rules from compacted Kafka Topic - open() vs Connected Streams

What i was trying to achieve from above was similar to GlobalKTable in Kafka Streams. https://cwiki.apache.org/confluence/display/KAFKA/KIP-99%3A+Add+Global+Tables+to+Kafka+Streams Also current flink version i am using is 1.4 Are there any other suggestions/guidance to achieve GlobalKTable functio

flink 1.4.2 Ambari

Hello I am trying to find the way to add Flink 1.4.2 service to ambari because is not listed in the Stack. does anyone has the steps to add this service manually? Thank you Best regards

Re: Need assistance : creating remote environment

Updated akka's 'reference.conf' file in my app shadow jar. Working fine now. Thank you. On Fri, Jul 13, 2018 at 10:46 PM Mohan mohan wrote: > Hi Chesnay, > > Now server and client/IDE both are using 1.4.0. > When I submit job from IDE getting below exception, > > com.typesafe.config.ConfigExcep