Removing labels from Flink metrics

Hi Team, Is it possible to remove a few labels from Flink operator scope metrics as we are noticing that sometimes those labels are too large and hence causing unnecessary load at our monitoring platform. One such label is operator_name. Thanks and Regards , Surendra Lalwani -- IMPORTANT

Metrics exposed by flink containing long label values

Hi Team, We are exposing metrics by Flink to prometheus but we are seeing it contains various labels even including operator details. and task_name, do we have any way from which we can remove these extra / unnecessary labels? Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e

KarkaConsumer_records_lag_max metrics showing weird numbers

inside flink UI show lag as 0, can anybody tell me what could be the reason? Thanks, Surendra -- Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain confidential information and is intended only for the addressee(s) named above. If you

Performing left join between two streams

Hi Team, Is it possible in Flink to perform Left Outer join between two streams as it is possible in Spark. I checked for Internal join it only supports inner join as of now. Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain

Re: How to perform something on Checkpointing and Savepointing

Tried these as well but they need to be called with some MapFunction or something else, we cannot just apply this Listener to the ExecutionEnvironment itself? Thanks and Regards , Surendra Lalwani On Mon, Sep 12, 2022 at 8:42 PM Ben Edwards wrote: > CheckpointedFunction / CheckpointListe

How to perform something on Checkpointing and Savepointing

Hi Team, I want to perform some customized operations on checkpointing and savepointing. For example, I want to create an additional file every time checkpointing and savepointing gets triggered. Do we have anything as such? Any listeners or anything else? Thanks and Regards , Surendra Lalwani

Overhead on managing Timers with large number of keys

for those keys. Will there be any overhead and if yes what will be the overhead, will it impact processing time? Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain confidential information and is intended only for the addressee(s

Best way to perform look up with historical data in Flink

helpful. Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain confidential information and is intended only for the addressee(s) named above. If you are not the intended recipient(s), you should not disseminate, distribute, or copy this e-mail

Re: How to avoid multiple reading from same topic

using just a single read. Thanks and Regards , Surendra Lalwani On Tue, May 17, 2022 at 1:19 PM Caizhi Weng wrote: > Hi! > > I wanted to explore if we can avoid reading multiple times and read only >> once >> > > Could you elaborate more on the reason you need this? If

How to avoid multiple reading from same topic

. Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain confidential information and is intended only for the addressee(s) named above. If you are not the intended recipient(s), you should not disseminate, distribute, or copy this e-mail

Data loss while writing to File System

Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain confidential information and is intended only for the addressee(s) named above. If you are not the intended recipient(s), you should not disseminate, distribute, or copy this e-mail. Please notify

Re: table.local-time-zone not working

I have also tried to set it in flink-conf.yaml, it is visible in Job Manager Configurations but still NOW() and PROCTIME() are returning values in UTC. Thanks and Regards , Surendra Lalwani On Tue, May 10, 2022 at 1:09 PM Surendra Lalwani wrote: > Hi Leonard, > > Flink Version is 1.1

Re: table.local-time-zone not working

s")); Thanks and Regards , Surendra Lalwani On Tue, May 10, 2022 at 1:07 PM Leonard Xu wrote: > Hi, Lalwani > > Could you share how you set this property and your Flink version? > > Best, > Leonard > > > 2022年5月10日 下午3:01，Surendra Lalwani 写道： > > Hi Team, > &

table.local-time-zone not working

Hi Team, I have tried using this property table.local-time-zone But it seems like it is not making any impact, on calling functions like PROCTIME() and NOW(), it always returns UTC value. Any help would be appreciated. Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail

Re: How to return JSON Object from UDF

Hi Timo, Please find the example: @DataTypeHint(value = "RAW", bridgedTo = JsonObject.class) public Object eval(String jsonString) { //Logic to parse } Thanks and Regards , Surendra Lalwani On Fri, May 6, 2022 at 3:13 PM Timo Walther wrote: > Can you show the full example?

Re: How to return JSON Object from UDF

r.java:233) ... 83 common frames omittedCaused by: org.apache.flink.table.api.ValidationException: Field 'threshold' of class 'net.minidev.json.JSONObject' is neither publicly accessible nor does it have a corresponding getter method. Thanks and Regards , Surendra Lalwani

How to return JSON Object from UDF

help would be appreciated. Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain confidential information and is intended only for the addressee(s) named above. If you are not the intended recipient(s), you should not disseminate, dist

HOP_PROCTIME is returning null

27; SECOND, INTERVAL '30' SECOND) Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain confidential information and is intended only for the addressee(s) named above. If you are not the intended recipient(s), you should not dissemi

Re: CVE-2021-44228 - Log4j2 vulnerability

Hi Team, Any updates on Flink 1.13.6 version release? Regards, Surendra Lalwani On Fri, Feb 4, 2022 at 1:23 PM Martijn Visser wrote: > Hi Surendra, > > You can follow the discussion on this topic in the Dev mailing list [1]. I > would expect it in the next couple of weeks. >

Re: CVE-2021-44228 - Log4j2 vulnerability

Hi Team, Any ETA on Flink version 1.13.6 release. Thanks and Regards , Surendra Lalwani On Sun, Jan 9, 2022 at 3:50 PM David Morávek wrote: > Flink community officially only supports current and previous minor > versions [1] (1.13, 1.14) with bug fixes. Personally I wouldn’t expect &

How to put external data in EmbeddedRocksDB

, does anyone of you have any idea on how this can be done or if anybody has any other solution, that is also appreciable. Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain confidential information and is intended only for the addressee(s

JsonRowSerializationSchema unable to parse TIMESTAMP_LTZ fields

Hi Team, JsonRowSerializationSchema is unable to parse fields with type TIMESTAMP_LTZ, seems like this is not handled properly. While trying to fire a simple query Select current_timestamp from table_name , it gives error that Could not serialize row and asks to add shaded flink dependency for jsr