Re: Query on Flink SQL create DDL primary key for nested field

Hi Xuyang, Thank you for your response. Since, I have no access to create a ticket in the ASF jira I have requested for the access and once I get the access will raise a ticket for the same. Also, you have asked me to use Datastream API to extract the id and then use the TableAPI feature, since I

Re: [DISCUSS][FLINK-33240] Document deprecated options as well

Thanks for the proposal. +1 from my side and +1 for putting them to a separate section. Best, Hang Samrat Deb 于2023年11月1日周三 15:32写道： > Thanks for the proposal , > +1 for adding deprecated identifier > > [Thought] Can we have seperate section / page for deprecated configs ? Wdut > ? > > > Bests

Re:Re: Query on Flink SQL create DDL primary key for nested field

Hi, Elakiya, I think you can get what you need here[1] with many examples briging DataStream api and Table API. There may be some redundancy, and I'm not sure this is a best way to resolve the question. First, use the StreamTableEnvironment to execute your original ddl without pk. Second, us

Re: Re: Query on Flink SQL create DDL primary key for nested field

Hi Xuyang, Thanks again for giving me some insights on how to use the Datastream API for my use case, I will explore it and experiment with it. I wanted to use the value inside the row datatype as a primary key because, I might get multiple records for the same id and when I try to make a join wi

Re: [DISCUSS][FLINK-33240] Document deprecated options as well

Hi Alexander, I haven't done a complete analysis yet. But through simple code search, roughly 35 options would be added with this change. Also note that some old options defined in a ConfigConstant class won's be added here as flink-doc won't discover these constant-based options. Best, Zhangh

Re: [DISCUSS][FLINK-33240] Document deprecated options as well

Hi Samrat and Ruan, Thanks for the suggestion. I'm actually in favor of adding the deprecated options in the same section as the non-deprecated ones. This would make user search for descriptions of the replacement options more easily. It would be a different story for options deprecated because

Re: Issues about removed topics with KafkaSource

Hi, That's by design: you can't dynamically add and remove topics from an existing Flink job that is being restarted from a snapshot. The feature you're looking for is being planned as part of FLIP-246 [1] Best regards, Martijn [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageI

Re: Monitoring File Processing Progress in Flink Jobs

Hi arjun Flink will save the currently processed file and its corresponding offset in Flink state [1]. You may need to use the Flink state process API[1] to access it. However, I don't think this is a good approach. I suggest adding relevant metrics to the FileSystem connector to report the curre

Fwd: flink-kubernetes-operator cannot handle SPECCHANGE for 100+ FlinkDeployments concurrently

Hi Flink Community, I am currently running flink-kubernetes-operator 1.6-patched ( https://github.com/apache/flink-kubernetes-operator/commit/3f0dc2ee5534084bc162e6deaded36e93bb5e384), and I have 3 flink-kubernetes-operator pods running. Recently, I deployed around 110 new FlinkDeployments, and I

Re: Issues about removed topics with KafkaSource

Hey Martijn Thanks for the clarification. Now it makes sense. I saw this feature FLIP-246 is still a WIP and there's no release date yet, and it actually contains quite some changes in it. We noticed there's a WIP PR for this change, just wondering if there's any plan in releasing this feature?