Re: Flink Task/Operator metrics renaming

2021-09-12 Thread Guowei Ma
Hi, Ashutosh As far as I know, there is no way to rename the system metrics name. But would you like to share why you need to rename the metrics ? Best, Guowei On Mon, Sep 13, 2021 at 2:29 PM Ashutosh Uttam wrote: > Hi team, > > We are using PrometheusReporter to expose Flink metrics to Prome

Re: TaskManagers OOM'ing for Flink App with very large state only when restoring from checkpoint

2021-09-12 Thread Guowei Ma
Hi, Kevin 1. Could you give me some specific information, such as what version of Flink is you using, and is it using DataStream or SQL? 2. As far as I know, RocksDB will put state on disk, so it will not consume memory all the time and cause OOM in theory. So you can see if there are any obje

Flink Task/Operator metrics renaming

2021-09-12 Thread Ashutosh Uttam
Hi team, We are using PrometheusReporter to expose Flink metrics to Prometheus. Is there possibility of renaming Task/Operators metric like numRecordsIn, numRecordsOut etc. and exposing it to Prometheus. Regards, Ashutosh

Re: Issue while creating Hive table from Kafka topic

2021-09-12 Thread Harshvardhan Shinde
Hi, I checked for the dependency using the stackoverflow link you sent, the ByteArrayDeserializer class is present in my jar but when I try to run it, I'm getting the same error message. I also removed the kafka and kafka-client dependencies from my pom.xml file and added the jar from the link you

Re: Documentation for deep diving into flink (data-streaming) job restart process

2021-09-12 Thread Puneet Duggal
Hi Robert, Any solution / alternate approach to above issue would be appreciated as going live with new jobs will be unreliable w.r.t task manager going down. On Fri, Sep 10, 2021 at 1:17 PM Puneet Duggal wrote: > Hi Robert, > > Thanks for taking out time to go through the logs. > > Problem: >

[no subject]

2021-09-12 Thread chang li