Queryable state support in Flink 1.9

I was testing with Flink 1.9. Here is how I set up mini cluster int port = 6124; int parallelism = 2; Configuration config = new Configuration(); config.setInteger(JobManagerOptions.PORT, port); config.setString(JobMan

Re: Timestamp Watermark Assigner bpund question

Hi, Vijay >>>Then the Operator progresses to the next Watermark as a starting point for events after event time reaches currWatermark ? AFAIK, the operator that generates watermark is called by the frame work. When the operator is called depends on the operator itself. For example the operator tha

Re: Join of DataStream and DataSet

Hi Reminia, Currently, we can't join a DataStream with a DataSet in Flink. However, the DataSet is actually a kind of bounded stream. From the point of this view, you can use a streaming job to achieve your goal. Flink Table API & SQL support different kinds of join[1]. You can take a closer look

Re: Timestamp Watermark Assigner bpund question

sorry for missing a not. :( Whether the watermark, which is generated by the AssignerWithPunctuatedWatermarks/AssignerWithPeriodicWatermarks is send to the downstream is controlled by the framework. If an operator returns a watermark going back Flink would _*not_* send it to the downstream. Best,

Can back pressure data be gathered by Flink metric system?

Hi Experts, From the page Flink metric system(https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/metrics.html#system-metrics ), I do not find the info about the back p

Re: Netty channel closed at AKKA gated status

Hi Wenrui, I think the akka gated issue and inactive netty channel are both caused by some task manager exits/killed. You should double check the status and reason of this task manager `'athena592-phx2/10.80.118.166:44177'`. Best, Zhijiang ---

Re: Retain metrics counters across task restarts

Hi Peter, The lifecycle of these metrics are coupled with lifecycle of task, So the metrics would be initialized after task is restarted. I think of one possible option is that you could store your required metrics into state, then the metric states would be restored from backend after task is

Re: Can back pressure data be gathered by Flink metric system?

Hi Henry, The backpressure tracking is not realized in metric framework, you could check the details via [1]. I am not sure why your requirements is showing backpressure in metrics. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/back_pressure.html Best, Zhijiang

Re: How would I use OneInputStreamOperator to deal with data skew?

Hi, You can checkout the bundle operator which used in Blink to perform similar thing you mentioned: https://github.com/apache/flink/blob/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/runtime/bundle/BundleOperator.java Best, Kurt On Fri, Apr 12, 2019 at 8:05 PM Felipe G

Re: Queryable state support in Flink 1.9

Hi, I think you should check TM log first and check if there are some info like: 1430 [main] INFO org.apache.flink.queryablestate.server.KvStateServerImpl - Started Queryable State Server @ /127.0.0.1:9069. 1436 [main] INFO org.apache.flink.queryablestate.client.proxy.KvStateClientProxyImpl - Sta

programmaticly start yarn session for flink job

Hi everyone! Is it a good way to start a yarn session programmatically for some flink jobs under kerbores? Thank you very much!

RE: Flink JDBC: Disable auto-commit mode

Hi Rong, We have already tried to set the fetch size with no success. According to PG documentation we have to set both configuration parameters (i.e., auto-commit to false and limit fetch) to achieve our purpose. Thanks, Konstantinos From: Rong Rong Sent: Παρασκευή, 12 Απριλίου 2019 6:50 μμ