Re: support escaping `#` in flink job spec in Flink-operator

It is also possible that this is a problem of the Flink native Kubernetes integration, we have to check where exactly it goes wrong before we try to fix it . We simply set the args into a Flink config and pass it to the native deployment logic in the operator. Gyula On Tue, 8 Nov 2022 at 07:37,

Re: support escaping `#` in flink job spec in Flink-operator

Hi! How do you submit your yaml? It’s possible that this is not operator problem. Did you try submitting the deployment in json format instead? If it still doesn't work please open a JIRA ticket with the details to reproduce and what you have tried :) Cheers Gyula On Tue, 8 Nov 2022 at 04:56,

Re: [blog article] Howto migrate a real-life batch pipeline from the DataSet API to the DataStream API

Hi Etienne, Very thanks for the article! Flink is currently indeed keeping increasing the ability of unified batch / stream processing with the same api, and its a great pleasure that more and more users are trying this functionality. But I also have some questions regarding some details. First IM

FeatHub - A feature store that uses Apache Flink for real-time feature ETL

Hi everyone, In case some of you might be interested, we (developers at Alibaba Flink team) recently developed and open-sourced FeatHub , a feature store that uses Apache Flink for feature engineering. The github page https://github.com/alibaba/feathub provides

support escaping `#` in flink job spec in Flink-operator

Hi, We have a job that contains `#` as part of mainArgs and it used to work on Ververica. Now we are switching to our own control plane to deploy to flink-operaotor and the job started to fail due to the main args string getting truncated at `#` character when passed to flink application. I believ

Re: [blog article] Howto migrate a real-life batch pipeline from the DataSet API to the DataStream API

Thanks for your post, It looks very good to me, also maybe for developers, Best, Liudalong yuxia 于2022年11月8日周二 09:11写道： > Wow, cool! Thanks for your work. > It'll be definitely helpful for the users that want to migrate their batch > job from DataSet API to DataStream API. > > Best regards, >

Kinesis Connector does not work

Hi, I'm following the kinesis connector instructions as documented here: https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/connectors/datastream/kinesis/ I'm also running Flink in standalone session mode using docker compose and the Python images, as described in the Flink docs (Depl

Re: [blog article] Howto migrate a real-life batch pipeline from the DataSet API to the DataStream API

Wow, cool! Thanks for your work. It'll be definitely helpful for the users that want to migrate their batch job from DataSet API to DataStream API. Best regards, Yuxia - 原始邮件 - 发件人: "Etienne Chauchot" 收件人: "dev" , "User" 发送时间: 星期一, 2022年 11 月 07日 下午 10:29:54 主题: [blog article] Howto m

[blog article] Howto migrate a real-life batch pipeline from the DataSet API to the DataStream API

Hi everyone, In case some of you are interested, I just posted a blog article about migrating a real-life batch pipeline from the DataSet API to the DataStream API: https://echauchot.blogspot.com/2022/11/flink-howto-migrate-real-life-batch.html Best Etienne

Load Distribution in specific Slot of Taskmanager in flink(version 1.15.2)

Dear Team, I need some advice on setting up load distribution of flink tasks. I have a flink task that processes transactions for users. Since load is more than what can be handled on single machine, I want same task to be executed on 3 machines. I am trying to use parallelism feature

Re: Consumption stops on some Kafka partitions in job running on Flink 1.15.2

Say we have two partitions A and B. A is the partition which is not receiving any events, but B is seeing events and processing normally. Now, because A is not having any events, with no Watermarking strategy, it is holding the watermark back as described in the docs. Now, does this mean that lag o

Re: Consumption stops on some Kafka partitions in job running on Flink 1.15.2

Hi Samuel, I'm glad to hear that! Let us know how the problem is finally solved. Personally I'd upgrade to 1.15.3. Salva On Mon, Nov 7, 2022 at 9:42 AM Samuel Chase wrote: > Hi Salva, > > Thanks for the pointers. They were helpful in gaining a better > understanding what happened. > > In both

Re: Consumption stops on some Kafka partitions in job running on Flink 1.15.2

Hi Salva, Thanks for the pointers. They were helpful in gaining a better understanding what happened. In both situations, these outages occurred at a time of the lowest traffic in a day. Due to business-logic reasons, we are using a partition key which may not result in even distribution across a