question regarding metadatahandlerProvider

Hi, I am using apache-flink 1.19 and python 3.11. I have a very simple batch job which registers a source table using CREATE TABLE … and output to a sink table, another CREATE TABLE …. Before I output to sink table, I run a dedup query SELECT * FROM ( SELECT *, ROW_NUMBER() OVER (PARTITION BY

Re: Help regarding Flink Stateful API

Nitin Chauhan 5:21 PM (6 minutes ago) to Andreas Hello Gabor, Thanks for the response. I have a set of specific questions, it could be of great help if you could answer them 1) Can stateful functions be executed only using Docker images provided by flink? I could not execute it as a standalone

Re: Help regarding Flink Stateful API

As I see you use stateful functions. Please consider whether your needs and possibilities match the actual stand of statefun project: https://lists.apache.org/thread/7cr2bgt91ppk6pz8o0nfbd10gs63nz6t G On Tue, Oct 1, 2024 at 1:59 PM Nitin Chauhan wrote: > Nitin Chauhan > 5:21 PM (6 minutes ago

Help regarding Flink Stateful API

HI I am trying to run a code using stateful APIs in my local machine. I am able to run it if I use docker image provided by flink official source code. I wanted to know is there any way in which we can do it without using the docker image and configuring everything on my own? Best Regards, Nitin

Re: Help regarding Flink Stateful API

Thanks for the help Let me go through the link On Tue, Oct 1, 2024 at 6:09 PM Gabor Somogyi wrote: > As I see you use stateful functions. > Please consider whether your needs and possibilities match the actual > stand of statefun project: > https://lists.apache.org/thread/7cr2bgt91ppk6pz8o0nfbd

Re: Help regarding Flink Stateful API

Hi Nitin, Flink applications can be started locally (run the main) for example from Intellij or any other similar IDE. Important note that such case the execution path is different but it's convenient for business logic debugging. BR, G On Tue, Oct 1, 2024 at 12:21 PM Nitin Chauhan wrote: > H

Re: [External] Regarding java.lang.IllegalStateException

My guess it’s a major known issue. Need a workaround. https://issues.apache.org/jira/browse/FLINK-32212 /Maxim From: prashant parbhane Date: Tuesday, April 23, 2024 at 11:09 PM To: user@flink.apache.org Subject: [External] Regarding java.lang.IllegalStateException Hello, We have been facing

Re: Regarding java.lang.IllegalStateException

tor 1.9.0 . . . . . . . . . . . . . . . . . . . . . Maxim Senin Senior Backend Engineer COGILITY<http://cogility.com/> From: prashant parbhane Date: Tuesday, April 23, 2024 at 11:09 PM To: user@flink.apache.org Subject: Regarding java.lang.IllegalStateException Hello, We have been facing this weird issu

Regarding java.lang.IllegalStateException

Hello, We have been facing this weird issue, where we are getting below exception and the job is getting restarted with new task managers. We are using flink 1.17. Same job works fine with lower number of task managers. (<10) java.lang.IllegalStateException: The library registration references a

Re: Inquiry Regarding Flink Tumbling Window Persistence and Restart Handling for File Source

Hi Arjun, Thanks for your query. Flink is fault tolerant and supports exactly-once semantics. In your case, the aggregated values can be recovered in case of a failure or application restart. You just need to enable checkpointing and configure an appropriate state backend. Regards, Jeyhun > > O

Inquiry Regarding Flink Tumbling Window Persistence and Restart Handling for File Source

Hello team, I'm relatively new to Flink's window functions, and I've configured a tumbling window with a 10-minute duration. I'm wondering about the scenario where the Flink job is restarted or the Flink application goes down. Is there a mechanism to persist the aggregated values, allowing the pro

Re: Instructions / steps regarding contributing : flink-training

Hello Milind, Thank you for your interest in contributing to flink-training! To add or modify unit tests, please start by creating a JIRA ticket at the FLINK JIRA page: https://issues.apache.org/jira/browse/FLINK, and set the "Component/s" to "Documentation / Training / Exercises" for your ticket.

Instructions / steps regarding contributing : flink-training

Hi I want to add / modify a few unit tests for flink-training. I have came across following instructions : Contributing I am not sure about the formal process of submitting the change or any JIRA ticket etc requirements.

Query Regarding CSV Decoder

Hi Team, I am using CSV decoder with Flink file source. I am stuck with decoding issues as below- 1. In case there is any blank line in between two records or blank lines in the end of file, it returns the blank object. E.g- Input Records: id,name,age,isPermanent,tenure,salary,gender,contact

Re: Question regarding asyncIO timeout

Hi, Leon > Besides that, Do you know if the async timeout is actually a global timeout? meaning it accounts for the time of each attempt call plus any interval time in between. Yes, the timeout is total timeout, you can see [1][2] for more detail. [1] https://cwiki.apache.org/confluence/pages/v

Re: Question regarding asyncIO timeout

Hi Ken, Thanks for the suggestion. Definitely a good call to just wrap the retry inside the client code. I'll give it a try. Besides that, Do you know if the async timeout is actually a global timeout? meaning it accounts for the time of each attempt call plus any interval time in between. I incre

Re: Question regarding asyncIO timeout

Hi Leon, Normally I try to handle retrying in the client being used to call the server, as you have more control/context. If that’s not an option for you, then normally (un)orderedWaitWithRetry() should work - when you say “it doesn’t seem to help much”, are you saying that even with retry you

Question regarding asyncIO timeout

Hi Flink users, We are using Flink AsyncIO to call a grpc-based service in our Flink job. And from time to time we are experiencing Async function timeout issues, here's the exception. ``` java.lang.Exception: Could not complete the stream element: Record @ 169393916 : [B@cadc5b3. Caused by: j

Query Regarding Optimisation of Timer Management in Flink CEP (Version 1.17)

Hi, I am currently working with Flink CEP version 1.17, and I am in the process of load testing for potential memory leaks related to checkpoint data. While analyzing the CepOperator code, I have come across a particular pattern regarding timer registration and event processing that I believe

Re: 回复: Questions regarding adaptive scheduler with YARN and application mode

wiki.apache.org/confluence/display/FLINK/FLIP-291%3A+Externalized+Declarative+Resource+Management> > [3] Autoscaler | Apache Flink Kubernetes Operator > <https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.5/docs/custom-resource/autoscaler/> > > Best, > Zha

Re: 回复: Questions regarding adaptive scheduler with YARN and application mode

n[3] Autoscaler | Apache Flink Kubernetes Operator Best,Zhanghao Chen发件人: Leon Xu 发送时间: 2023年6月27日 13:41 收件人: user 主题: Questions regarding adaptive scheduler with YARN and application mode Hi Flink users, I am trying to use Adaptive Scheduler to auto scale our Flink streaming jobs (NOT batch

回复: Questions regarding adaptive scheduler with YARN and application mode

est, Zhanghao Chen 发件人: Leon Xu 发送时间: 2023年6月27日 13:41 收件人: user 主题: Questions regarding adaptive scheduler with YARN and application mode Hi Flink users, I am trying to use Adaptive Scheduler to auto scale our Flink streaming jobs (NOT batch job). Our jobs are running on YARN wit

Questions regarding adaptive scheduler with YARN and application mode

Hi Flink users, I am trying to use Adaptive Scheduler to auto scale our Flink streaming jobs (NOT batch job). Our jobs are running on YARN with application mode. There isn't much doc around how adaptive scheduler works. So I have some questions: 1. How does Adaptive Scheduler work with YARN/A

Re: Queries/Help regarding limitations on File source

Hi Kirti For the watermark problem, I think the description in the document mainly refers to the out-of-order data between multiple files. This will result in a large number of late events [1], which will generate a large number of retract events, and late events out of time will be discarded. [1

Queries/Help regarding limitations on File source

Hi, I am using Data stream file source connector in one of my use case. I was going through the documentation where I found below limitations: https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/filesystem/#current-limitations 1. Watermarking does not work very wel

Question regarding Flink-CDC for postgres

Hi Flink Users, We are looking to use Flink Postgres CDC but then we noticed it only supports single thread reading during the snapshot reading phase. Is there any plan to support parallel snapshot reading for postgres in the near future? Looks like this is not an issue for mysql. Thanks Leon

Question regarding java.lang.NoSuchMethodError in flink-s3-fs-hadoop.jar

Hi Flink Users, I am getting this exception after upgrading flink-s3-fs-hadoop from 1.15.3 to 1.16.0. *java.lang.NoSuchMethodError: 'com.google.protobuf.Descriptors$FileDescriptor org.apache.hadoop.security.proto.SecurityProtos.getDescriptor()'* After looking into the content of the flink-s3-fs-

Re: Regarding Changes in Flink Operator 1.3.1

Thanks a lot. On Thu, Jan 19, 2023, 1:08 PM Gyula Fóra wrote: > Please see the release announcements: > > https://flink.apache.org/news/2022/10/07/release-kubernetes-operator-1.2.0.html > > https://flink.apache.org/news/2022/12/14/release-kubernetes-operator-1.3.0.html > > https://flink.apache.o

Re: Regarding Changes in Flink Operator 1.3.1

Please see the release announcements: https://flink.apache.org/news/2022/10/07/release-kubernetes-operator-1.2.0.html https://flink.apache.org/news/2022/12/14/release-kubernetes-operator-1.3.0.html https://flink.apache.org/news/2023/01/10/release-kubernetes-operator-1.3.1.html https://nightlies.ap

Re: Regarding Changes in Flink Operator 1.3.1

also are the changes in operator version 1.3.1 backward compatible ? On Thu, Jan 19, 2023 at 12:38 PM Sumit Aich wrote: > Hi Team, > > Can you please share what has changed in Flink Kubernetes Operator version > 1.3.1 from the 1.1.0 version. > > Thanks, > Sumit >

Regarding Changes in Flink Operator 1.3.1

Hi Team, Can you please share what has changed in Flink Kubernetes Operator version 1.3.1 from the 1.1.0 version. Thanks, Sumit

Re: Regarding Flink Upgrades

Hi, What you linked to is what the community agreed to support. So far we've been able to support three versions at all times (e.g. currently we merge bugfixes to 1.17.x, 1.16.x, 1.15.x), which is one extra version than what is described in the docs. I don't think this will ever decrease. As

Re: Regarding Flink Upgrades

Hello Prasanna, 1) Of course we would always recommend you keep up to date. To receive support and fixes from the Flink community you should try to stick to the current/previous minor version, as per the policy. Releases for older versions are rare and typically only performed under exceptional ci

Regarding Flink Upgrades

Hi Community, Currently we are using version 1.12.7 and it is running without any issue. And we see that version 1.17 is set to release early next year. That means we would be 5 versions behind. 1) So how far can we lag behind the current flink version ? 2) If we face any issues like log4j that

Re: Question regarding to debezium format

Hello Edwin, Would you mind sharing a simple FlinkSQL DDL for the table you are creating with the kafka connector and dthe debezium-avro-confluent format? Also, can you elaborate on the mechanism who publishes initially to the schema registry and share the corresponding schema? In a nutshell, th

Re: Question regarding to debezium format

Hi Edwin, I'm suspecting that's because those fields are considered metadata which are treated separately. There's https://issues.apache.org/jira/browse/FLINK-20454 for adding the metadata support for the Debezium format with a PR provided, but not yet reviewed. If you could have a look at the PR

Question regarding to debezium format

Hi guys, I was trying to use flink sql to consume data from kafka source, the format of which is debezium-avro-confluent. And I encountered a AvroTypeException saying that "Found something, expecting union", where something is not a type but a field that I defined in the schema registery. So

Re: Query regarding akka.framesize in Flink

Hey, Akka frame maximum size is 2GB, which is limited by maximum java byte[] size. Not sure why you config is being rejected. If your Akka frames are getting large you can consider reducing state.storage.fs.memory-threshold [1],l. If you are using RocksDB with incremental checkpoints, triggering co

Query regarding akka.framesize in Flink

Hi Team, I am trying to set the value of akka.framesize in flinkconf.yaml to 2 gb, because of which I am getting the below exception which staring flink: "Caused by: java.lang.IllegalArgumentException: requirement failed: Setting 'maximum-frame-size' must be at least 32000 bytes" Flink is getti

[NOTICE] Blog post regarding Akka's licensing change

Hello, You may have heard about a recent change to the licensing of Akka. We just published a blog-post regarding this change and what it means for Flink. https://flink.apache.org/news/2022/09/08/akka-license-change.html TL;DR: Flink is not in any immediate danger and we will ensure that

Re: Question Regarding State Migrations in Ververica Platform

+dev > On Aug 30, 2022, at 11:20 AM, Rion Williams wrote: > >  > Hi all, > > I wasn't sure if this would be the best audience, if not, please advise if > you know of a better place to ask it. I figured that at least some folks here > either work for Ververica or might have used their platfor

Question Regarding State Migrations in Ververica Platform

Hi all, I wasn't sure if this would be the best audience, if not, please advise if you know of a better place to ask it. I figured that at least some folks here either work for Ververica or might have used their platform. *tl;dr; I'm trying to migrate an existing stateful Flink job to run in Verv

Questions regarding JobManagerWatermarkTracker on AWS Kinesis

Hi there! I have a Flink Job (v 1.13.2, AWS managed) which reads from Kinesis (AWS manger, 4 shards). For reasons the shards are not partitioned properly (at the moment). So I wanted to make use of Watermarks (BoundedOutOfOrdernessTimestampExtractor) and the JobManagerWatermarkTracker to avoid

Re: New KafkaSource API: Change in default behavior regarding starting offset

15/2022 17:50 > To Martijn Visser > Cc Jing Ge , > user > Subject Re: New KafkaSource API : Change in default behavior regarding > starting offset > Hello Martijn, > > Thanks for the link to the release note, especially : > "When resuming f

Re: New KafkaSource API: Change in default behavior regarding starting offset

请问这个邮件咋退订？ Replied Message | From | bastien dine | | Date | 06/15/2022 17:50 | | To | Martijn Visser | | Cc | Jing Ge, user | | Subject | Re: New KafkaSource API : Change in default behavior regarding starting offset | Hello Martijn, Thanks for the link to the release note

Re: New KafkaSource API : Change in default behavior regarding starting offset

Hello Martijn, Thanks for the link to the release note, especially : "When resuming from the savepoint, please use setStartingOffsets(OffsetsInitializer.committedOffsets()) in the new KafkaSourceBuilder to transfer the offsets to the new source." So earliest is the new default We use for sure .co

Re: New KafkaSource API : Change in default behavior regarding starting offset

Hi Bastien, When the FlinkKafkaConsumer was deprecated in 1.14.0, the release notes included the instruction how to migrate from FlinkKafkaConsumer to KafkaConsumer [1]. Looking at the Kafka documentation [2], there is a section on how to upgrade to the latest connector version that I think is out

Re: New KafkaSource API : Change in default behavior regarding starting offset

Hello jing, This was the previous method in old Kafka consumer API, it has been removed in 1.15, so source code is not in master anymore, Yes I know for the new Offset initializer, committed offset + earliest as fallback can be used to have the same behavior as before I just wanted to know whether

Re: New KafkaSource API : Change in default behavior regarding starting offset

Hi Bastien, Thanks for asking. I didn't find any call of setStartFromGroupOffsets() within Flink in the master branch. Could you please point out the code that committed offset is used as default? W.r.t. the new KafkaSource, if OffsetsInitializer.committedOffsets() is used, an exception will be t

New KafkaSource API : Change in default behavior regarding starting offset

Hello everyone, Does someone know why the starting offset behaviour has changed in the new Kafka Source ? This is now from earliest (code in KafkaSourceBuilder), doc says : "If offsets initializer is not specified, OffsetsInitializer.earliest() will be used by default." from : https://nightlies.a

Re: Questions regarding classpath loading order in YarnClusterDescriptor

o Geng From: Leon Xu Date: Sunday, June 5, 2022 at 4:04 PM To: Biao Geng Cc: user Subject: Re: Questions regarding classpath loading order in YarnClusterDescriptor Hi Biao, I really appreciate your thorough answers. And yes for now I took the workaround by manipulating the directory names. To

Re: Questions regarding classpath loading order in YarnClusterDescriptor

Hi Biao, I really appreciate your thorough answers. And yes for now I took the workaround by manipulating the directory names. To follow up with one more question if you don't mind: What is the recommended way of managing plugins in YarnClusterDescriptor? Currently I am placing the plugins (e.g. f

Re: Questions regarding classpath loading order in YarnClusterDescriptor

Hi Leon, For your question1, in the classpath, there are 2 types of jars: user jars and flink system jars(i.e. jars in flink/lib). System jars are sorted alphabetically. For user jars, there are 3 choices to add user jars in the final classpath: ORDER, FIRST, LAST(See the doc

Questions regarding classpath loading order in YarnClusterDescriptor

Hi Flink Community, We are building on top of *org.apache.flink.yarn.YarnClusterDescriptor *to submit a flink application from Java code to YARN cluster, in the application mode. We are setting the classpath as the value of *the yarn.provided.lib.dirs *property under the yarn configuration. By p

Re: Query regarding ClassCastException

don't have experience with Avro caching behavior; the first two suggestions are probably more relevant there, according to the docs. Best, Nico On Fri, Mar 25, 2022 at 5:22 AM Mahima Agarwal wrote: > Hi Team, > > Just a follow up mail regarding the query. > > Any leads would

Re: Query regarding ClassCastException

Hi Team, Just a follow up mail regarding the query. Any leads would be appreciated. Thanks and Regards Mahima Agarwal On Mon, Mar 21, 2022 at 2:56 PM Mahima Agarwal wrote: > Hi Team, > > I am getting a ClassCastException in my flink job's business logic when > using the be

Query regarding ClassCastException

Hi Team, I am getting a ClassCastException in my flink job's business logic when using the below code to clone an object with newBuilder. MyClass outEvent = new MyClass(); outEvent = (MyClass) MyClass.newBuilder(inEvent).build(); Exception : Caused by: java.lang.ClassCastException: com.avro.mon.

Re: [External] Require help regarding possible issue/bug I'm facing while using Flink

Hi De Xun, Unfortunately MAP, ARRAY and ROW types are supported by Flink Parquet format only since Flink 1.15 (see FLINK-17782 [1], not released yet). You may want to upgrade Flink version to 1.15 once it is released, or make your own implementation based on the latest code on master branch fo

Re: Require help regarding possible issue/bug I'm facing while using Flink

Hi De Xun, I created an answer in the StackOverflow and hope it would be helpful. I’d like repost my answer here for the convenience of people in mailing lists. The first call of RowRowConverter::toInternal is an internal implementation for making a deep copy of the StreamRecord emitted by tab

Require help regarding possible issue/bug I'm facing while using Flink

Greetings, I'm facing a difficult issue/bug while working with Flink. Would definitely appreciate some official expert help on this issue. I have posted my problem on StackOverflow , but have no

Re: Questions regarding connecting local FlinkSQL client to remote JobManager in K8s

Hi Elkhan, I confirm that the FlinkSQL Client is communicating with JM via Rest endpoint. After I changed the “rest.port”, the sql client thrown exception: "[ERROR] Could not execute SQL statement. Reason: java.net.ConnectException: Connection refused”. So for your case, since Flink will creat

Re: Questions regarding connecting local FlinkSQL client to remote JobManager in K8s

Hi Elkhan, Except for JM have an external IP address, I think the port 6123 also need to be opened. You may need to set a host port for 6123 in JM pod or expose this port by Kubernetes service. But I am not sure whether the sql-client communicate with JM via Rest endpoint or RPC port. Hopes som

Questions regarding connecting local FlinkSQL client to remote JobManager in K8s

Hi Flink users, Wanted to check if any of you tried to run the local FlinkSQL client against JobManager running in the Kubernetes environment. For local FlinkSQL Client and local Flink cluster we set these params: jobmanager.rpc.address: localhost jobmanager.rpc.port: 6123 To make it work, Is t

Regarding delayed reading at source with larger checkpoint timeouts

Hi Team, I have a use case where in my Kafka Source, I need to wait for 2 hours before handling a event. Currently, following is the plan, kindly let me know if this would work without issues and any gotchas I need to be aware of. a) In Kafka consumer deserializer schema, look at the published ti

Re: regarding flink metrics

Your best bet is to create a custom reporter that does this calculation. You could either wrap the reporter, subclass is, or fork it. In any case, https://github.com/apache/flink/tree/master/flink-metrics/flink-metrics-datadog should be a good starting point. On 01/02/2022 13:26, Jessy Ping wr

regarding flink metrics

Hi Team, We are using datadog and its http reporter( packaged in flink image) for sending metrics from flink application. We do have a requirement for setting tags with values calculated at runtime for the custom metrics emitted from Flink. Currently, it is impossible to assign tags at runtime. Is

Re: Regarding Queryable state in Flink

t; >> >> >> Sincere greetings >> >> >> >> Thias >> >> >> >> >> >> *From:* Jessy Ping >> *Sent:* Montag, 24. Januar 2022 16:47 >> *To:* user >> *Subject:* Regarding Queryable state in Flink >> >&

Re: Regarding Queryable state in Flink

cessor api [1] for offline analysis > of checkpoints and savepoints? > > > > [1] > https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/libs/state_processor_api/ > > > > Sincere greetings > > > > Thias > > > > > > *From:* Je

RE: Regarding Queryable state in Flink

: user Subject: Regarding Queryable state in Flink Hi Team, We are currently running our streaming application based Flink(Datastream API ) on a non-prod cluster.And planning to move it to production cluster soon.. We are keeping cerating keyed state backed by rocksdb in the flink application

Regarding Queryable state in Flink

Hi Team, We are currently running our streaming application based Flink(Datastream API ) on a non-prod cluster.And planning to move it to production cluster soon.. We are keeping cerating keyed state backed by rocksdb in the flink application. We need a mechanism to query these keyed state values

Re: Query regarding Kafka Source and Kafka Sink in 1.14.3

Hi! All properties you set by calling KafkaSource.builder().setProperty() will also be given to KafkaConsumer (see [1]). However these two properties are specific to Flink and Kafka does not know them, so Kafka will produce a warning message. These messages are harmless as long as the properties y

Query regarding Kafka Source and Kafka Sink in 1.14.3

Hi Team, I am trying to set the following properties in Kafka Source API in flink 1.14.3 version. -> client.id.prefix -> partition.discovery.interval.ms But I am getting the below mentioned warning in taskmanager logs: 1. WARN org.apache.kafka.clients.consumer.ConsumerConfig [] - Th

Re: Regarding Connector Options - value.deserializer

gt; > Ronak Beejawat > > > > > > > > *From:* Hang Ruan > *Sent:* Monday, January 10, 2022 3:06 PM > *To:* d...@flink.apache.org; Ronak Beejawat (rbeejawa) > *Cc:* commun...@flink.apache.org; user@flink.apache.org > *Subject:* Re: Regarding Connector Opti

Re: Regarding Connector Options - value.deserializer

Hi, Ronak, I think you should implement a custom format by yourself instead of overriding. The 'value.format' is a required table option. Best, Hang Ronak Beejawat (rbeejawa) 于2022年1月10日周一 17:09写道： > Hi Team, > > Is there any way we use value.deserializer in Connector Options from kafka > via

Re: Regarding the size of Flink cluster

y to spread the load more evenly. > any blogs regarding the results of Flink's load testing I would also recommend the FlinkForward YouTube channel. A lot of users stories including actual numbers and configurations are shown there. Regards, Timo [1] https://nightlies.apache.org/fli

Regarding the size of Flink cluster

Hi All, I have the following questions regarding the sizing of the Flink cluster doing stateful computation using Datastream API. It will be better if the community can answer the below questions or doubts. Suppose we have a pipeline as follows, *Kafka real time events source1 & Kafka r

Re: Query regarding exceptions API(/jobs/:jobid/exceptions)

>>>> >>>> [1] >>>> https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jobs-jobid-exceptions >>>> >>>> On Thu, Nov 25, 2021 at 12:14 PM Chesnay Schepler >>>> wrote: >>>> >>>>> root-

Re: Query regarding exceptions API(/jobs/:jobid/exceptions)

gt; >>>> root-exception: The last exception that caused a job to fail. >>>> all-exceptions: All exceptions that occurred the last time a job >>>> failed. This is primarily useful for completed jobs. >>>> exception-history: Exceptions that

Re: Query regarding exceptions API(/jobs/:jobid/exceptions)

All exceptions that occurred the last time a job failed. >>> This is primarily useful for completed jobs. >>> exception-history: Exceptions that previously caused a job to fail. >>> >>> On 25/11/2021 11:52, Mahima Agarwal wrote: >>> >>> Hi Team,

Re: Query regarding exceptions API(/jobs/:jobid/exceptions)

rily useful for completed jobs. >> exception-history: Exceptions that previously caused a job to fail. >> >> On 25/11/2021 11:52, Mahima Agarwal wrote: >> >> Hi Team, >> >> Please find the query below regarding exceptions >> API(/jobs/:jobid/exceptions) &

Re: Query regarding exceptions API(/jobs/:jobid/exceptions)

wrote: > > Hi Team, > > Please find the query below regarding exceptions > API(/jobs/:jobid/exceptions) > > > In response of above rest api: > > > Users are getting 3 types of exceptions: > 1. exceptionHistory > 2. all-exceptions > 3. root-exception > >

Re: Query regarding exceptions API(/jobs/:jobid/exceptions)

Team, Please find the query below regarding exceptions API(/jobs/:jobid/exceptions) In response of above rest api: Users are getting 3 types of exceptions: 1. exceptionHistory 2. all-exceptions 3. root-exception What is the purpose of the above 3 exceptions? Any leads would be

Query regarding exceptions API(/jobs/:jobid/exceptions)

Hi Team, Please find the query below regarding exceptions API(/jobs/:jobid/exceptions) In response of above rest api: Users are getting 3 types of exceptions: 1. exceptionHistory 2. all-exceptions 3. root-exception What is the purpose of the above 3 exceptions? Any leads would be

Re: Questions regarding broadcast join in Flink

Hi Gerald, actually, this is a typical issue when performing a streaming join. An ideal solution would be to block the main stream until the broadcast stream is ready. However, this is currently not supported in the API. In any case, a user needs to handle this in a use case specific way to

Questions regarding broadcast join in Flink

Hello, I am trying to implement a broadcast join of two streams in flink using the broadcast functionality. In my usecase I have a large stream that will be enriched with a much smaller stream. In order to first test my approach, I have adapted the Taxi ride exercise in the official training rep

Re: Queries regarding Flink upgrade strategies

t;>>> into the thread. He might have more insights to share. >>>>> >>>>> Best, >>>>> Matthias >>>>> >>>>> [1] >>>>> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/upgrading/#upg

Re: Queries regarding Flink upgrade strategies

gt;>> [1] >>>> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/upgrading/#upgrading-the-flink-framework-version >>>> >>>> On Thu, Aug 26, 2021 at 9:10 AM Amit Bhatia >>>> wrote: >>>> >>>>> Hi, &g

Re: Queries regarding Flink upgrade strategies

st, >>> Matthias >>> >>> [1] >>> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/upgrading/#upgrading-the-flink-framework-version >>> >>> On Thu, Aug 26, 2021 at 9:10 AM Amit Bhatia >>> wrote: >>> >>&

Re: Queries regarding Flink upgrade strategies

e are using Flink 1.13.2 with Kubernetes HA solution provided by flink. >>> We have created a deployment for JobManager and TaskManager with option to >>> deploy multiple replicas and the same is bundled in a single helm chart. >>> So we have below queries regarding Flink upg

Re: Queries regarding Flink upgrade strategies

HA solution provided by flink. >> We have created a deployment for JobManager and TaskManager with option to >> deploy multiple replicas and the same is bundled in a single helm chart. >> So we have below queries regarding Flink upgrade strategies, kindly help >> us to answer below

Re: Queries regarding Flink upgrade strategies

me is bundled in a single helm chart. > So we have below queries regarding Flink upgrade strategies, kindly help > us to answer below queries: > > 1) What upgrade strategies are supported by Flink (RollingUpdate/Recreate) > and which one is recommended for production use? > > 2)

Queries regarding Flink upgrade strategies

Hi, We are using Flink 1.13.2 with Kubernetes HA solution provided by flink. We have created a deployment for JobManager and TaskManager with option to deploy multiple replicas and the same is bundled in a single helm chart. So we have below queries regarding Flink upgrade strategies, kindly help

Re: Regarding state access in UDF

Hi Ingo, Thank you for your advice, we've not tried it yet, we just thought it may work that way, but now it seems not then. We'll see how it could match our use case with the AggregateFunction interface. On Thu, Jul 1, 2021 at 1:57 PM Ingo Bürk wrote: > Hi Kai, > > CheckpointedFunction is not

Re: Regarding state access in UDF

Hi Kai, CheckpointedFunction is not an interface meant to be used with UDFs (in the Table API / SQL sense[1]), but is rather an interface for DataStream API[2]; the term "user-defined function" has a different meaning there. Did you actually try it to see if it works? I'd be surprised it it did.

Re: Regarding state access in UDF

Hi Ingo, Thank you for the reply, we actually need more fine-grained control on the states in UDF. Per investigation, we found that the states can be simply created/accessed via implementing `CheckpointedFunction` interface, please advise if there is any side-effect by doing that. On Wed, Jun 30,

Re: Regarding state access in UDF

Hi Kai, AggregateFunction and TableAggregateFunction are both stateful UDF interfaces. This should cover most scenarios given where they would be used. If you need more fine-grained control you can also always drop down into the DataStream API (using #toDataStream) and work there. Table API / SQL

Regarding state access in UDF

Hi team, We've a use case that needs to create/access state in UDF, while per the documentation and UDF interface

Re: Regarding FLIP-91's status

Friday, May 28, 2021 5:18 AM To: Sonam Mandal Cc: user@flink.apache.org ; Jark Wu Subject: Re: Regarding FLIP-91's status Hi Sonam, It looks like it has been stale for some time. You might be able to restart the discussion replying to the respective thread in the dev mailing list [1]. You

Re: Regarding FLIP-91's status

Hi Sonam, It looks like it has been stale for some time. You might be able to restart the discussion replying to the respective thread in the dev mailing list [1]. You seem to be right about the repository based on Jark's reply in the related ticket FLINK-15472 [2]. I'm adding Jark to the thread. M

Regarding FLIP-91's status

Hello, I was curious about the progress on FLIP-91. Is this actively being developed? I believe the code is in development at https://github.com/ververica/flink-sql-gateway, is this the right REPO? I haven

  1   2   3   4   >