Re: Flink running same task on different Task Manager

2022-08-18 Thread Lijie Wang
Hi Great, -> Will these methods work? I think it will not work. It can control that the slots are evenly distributed on the TM, but cannot control the correspondence between tasks and slots. For example, the Flink cluster has 2 TMs, each TM has 4 slots(so total 8 slots in this cluster), and your j

Re: Is this a Batch SQL Bug?

2022-08-18 Thread Marco Villalobos
Sure. I created this ticket. First time creating a ticket. https://issues.apache.org/jira/browse/FLINK-29039 > On Aug 17, 2022, at 6:49 PM, yuxia wrote: > > Thanks for raising it. Yes, you're right. It's indeed a bug. > The problem is the RowData produced by LineBytesInputFormat is reused, bu

Re: Failing to maven compile install Flink 1.15

2022-08-18 Thread yuxia
which mvn version do you use? It's recommanded to use maven 3.2.5 Best regards, Yuxia 发件人: "hjw" <1010445...@qq.com> 收件人: "User" 发送时间: 星期四, 2022年 8 月 18日 下午 10:48:57 主题: Failing to maven compile install Flink 1.15 I try to maven clean install Flink 1.15 parent,but fail. A Error happene

Re: Failing to compile Flink 1.9 with Scala 2.12

2022-08-18 Thread yuxia
At least for Flink 1.15, it's recommended to use maven 3.2.5. So I guess maybe you can try use a lower version of maven. Best regards, Yuxia 发件人: "Milind Vaidya" 收件人: "Weihua Hu" 抄送: "User" 发送时间: 星期五, 2022年 8 月 19日 上午 1:26:45 主题: Re: Failing to compile Flink 1.9 with Scala 2.12 Hi W

Re: Flink Histogram not emitting sum while using Metrics Reporter

2022-08-18 Thread Chesnay Schepler
We currently do not compute sums for Histograms, and hence this isn't exported to Prometheus. You'd need to use custom histogram (that actually computes sums) and a custom prometheus reporter (that extracts said sum if present) to implement this yourself. On 18/08/2022 18:37, Sarang Vadali via

Re: Failing to compile Flink 1.9 with Scala 2.12

2022-08-18 Thread Milind Vaidya
Hi Weihua, Thanks for the update. I do understand that, but right now it is not possible to update immediately to 1.15, so wanted to know what is the way out. - Milind On Thu, Aug 18, 2022 at 7:19 AM Weihua Hu wrote: > Hi > Flink 1.9 is not updated since 2020-04-24, it's recommended to use the

Re: Flink running same task on different Task Manager

2022-08-18 Thread Great Info
Kindly help with this, I got stuck -> If so, I think you can set Task1 and Task2 to the same parallelism and set them in the same slot sharing group. In this way, Task1 and Task2 will be deployed into the same slot(That is, the same task manager). *Updating task details * *Task1- Source some stati

Flink Histogram not emitting sum while using Metrics Reporter

2022-08-18 Thread Sarang Vadali via user
Hi, I am currently registering a Flink Histogram and using the Prometheus Metrics Reporter to send this metric to our Time Series Data Storage. When Prometheus grabs this metric and converts it to the "summary" type, there is no sum found (only the streaming quantiles and count). This is causing

Failing to maven compile install Flink 1.15

2022-08-18 Thread hjw
I try to maven clean install Flink 1.15 parent,but fail. A Error happened in compiling flink-clients. Error Log: Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.4:single (create-test-dependency) on project flink-clients: Error reading assemblies: Error locating assembly d

Overwriting watermarks in DataStream

2022-08-18 Thread Peter Schrott
Hi there, While still struggling with events and watermarks out of order after sorting with a buffer process function (compare [1]) I tired to solve the issue by assigning a new watermark after the mentioned sorting function. The Flink docs [2] are not very certain about the impact of assigning

Re: Failing to compile Flink 1.9 with Scala 2.12

2022-08-18 Thread Weihua Hu
Hi Flink 1.9 is not updated since 2020-04-24, it's recommended to use the latest stable version 1.15.1. Best, Weihua On Thu, Aug 18, 2022 at 5:36 AM Milind Vaidya wrote: > Hi > > Trying to compile and build Flink jars based on Scala 2.12. > > Settings : > Java 8 > Maven 3.6.3 / 3.8.6 > > Many

Jobmanager sometimes exits at startup in kubernetes

2022-08-18 Thread Evgeniy Lyutikov
Hi all! I'm using Flink 1.14.4 along with kubernetes operator version 1.1.0, sometimes kubernetes operator restarts the cluster after changing the flinkdeployment object (with saving savepoint ), the new jobmanager which created exits right after start. 2022-08-18T06:47:52.627825838Z DEBUG o