Re: Parallelism under reactive scaling with slot sharing groups

2023-07-27 Thread Weihua Hu
Hi, Which Flink version are you using? I haven't reproduced this issue by using the master branch. Best, Weihua On Tue, Jul 25, 2023 at 2:56 AM Allen Wang wrote: > Hello, > > Our job has operators of source -> sink -> global committer. We have > created two slot sharing groups, one for sourc

RE: Custom TCP server socket source

2023-07-27 Thread Kamal Mittal via user
Hello Martijn, I followed the same link and created Enumerator but that enumerator looks like opens ServerSocket in task manager via custom source rather than at job manager. Attached the very rough custom source code. FlinkClientApp – Main program SSLServerCustomSocketStreamFunction_Copy – cus

Metrics are not reported in Python UDF (used inside FlinkSQL) when exception is raised

2023-07-27 Thread Elkhan Dadashov
Hi Flinkers, Wanted to check if anyone else has faced this issue before: When Python UDF (which is used inside FlinkSQL) raises an exception, then metrics get lost and not reported. Facing this issue both in Flink 1.16.2 and FLink 1.17.1 (Python 3.9). If an exception is not raised, then metrics

Re: Task manager creation in Flink native Kubernetes (application mode)

2023-07-27 Thread Mason Chen
I'm also curious about this and how to make it better in the current native Kubernetes integration model. Is there some way for Flink to discover and surface the oom kill signal from Kubernetes? Best, Mason On Tue, Jul 25, 2023 at 6:11 AM Alexis Sarda-Espinosa < sarda.espin...@gmail.com> wrote:

[ANNOUNCE] Apache Celeborn(incubating) 0.3.0 available

2023-07-27 Thread zhongqiang chen
Hi all, Apache Celeborn(Incubating) community is glad to announce the new release of Apache Celeborn(Incubating) 0.3.0 Celeborn is dedicated to improving the efficiency and elasticity of different map-reduce engines and provides an elastic, high-efficient service for intermediate data including s

Re: Custom TCP server socket source

2023-07-27 Thread Martijn Visser via user
Hi Kamal, The best starting point would be to look at how to write a custom source connector. Have a look at https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/sources/ which also includes links to the various classes that you'll need. Please let us know what else you've trie

Re: Unsubscribe

2023-07-27 Thread Martijn Visser
Hi, Please send an email to user-unsubscr...@flink.apache.org in order to be removed from the User mailing list. Best regards, Martijn On Wed, Jul 26, 2023 at 3:44 AM Lu Weizheng wrote: > Unsubscribe >