RE: Does reduce function on keyed window gives any guarantee on the order of elements?

2022-11-04 Thread Qing Lim
That’s my understanding as well, thanks for your confirmation. From: Yanfei Lei Sent: 04 November 2022 16:03 To: Qing Lim Cc: User Subject: Re: Does reduce function on keyed window gives any guarantee on the order of elements? Hi Qing, > am I right to think that there will be 1 reduce functio

Re: Does reduce function on keyed window gives any guarantee on the order of elements?

2022-11-04 Thread Yanfei Lei
Hi Qing, > am I right to think that there will be 1 reduce function per key, and they will never overlap? I agree with this, please correct me if I'm wrong. For a certain key, it will be divided into a certain key group range, and thus sent to a certain sub-task(reduce function instance). A key i

Re: UDFs classloading changes in 1.16

2022-11-04 Thread Alexander Fedulov
Hi Leonard, Sure, here is the new ticket: https://issues.apache.org/jira/browse/FLINK-29890 Best, Alexander Fedulov On Fri, Nov 4, 2022 at 2:12 PM Leonard Xu wrote: > Thanks Alexander for reporting this issue, Could you open a jira ticket as > well? > > CC: Shengkai, please take a look this ti

Re: question about Async IO

2022-11-04 Thread David Anderson
Yes, that will work as you expect. So long as you don't put another shuffle or rebalance in between, the keyed partitioning that's already in place will carry through the async i/o operator, and beyond. In most cases you can even use reinterpretAsKeyedStream on the output (so long as you haven't do

Re: UDFs classloading changes in 1.16

2022-11-04 Thread Leonard Xu
Thanks Alexander for reporting this issue, Could you open a jira ticket as well? CC: Shengkai, please take a look this ticket, looks like an incompatibility change. Best, Leonard > 2022年11月4日 下午6:15,Alexander Fedulov 写道: > > Hi everyone, > > 1.16 introduced quite a lot of changes with resp

UDFs classloading changes in 1.16

2022-11-04 Thread Alexander Fedulov
Hi everyone, 1.16 introduced quite a lot of changes with respect to classloading in the Table API. The way UDFs could previously be loaded from JARs in 1.15 does not work in 1.16 anymore - it fails with the ClassNotFound exception when UDFs are used at runtime. Here is a repository with a reprodu

Getting error stacktrace during job submission on Flink Operator

2022-11-04 Thread Meghajit Mazumdar
Hello folks, We in our team are currently running Flink clusters in Standalone, Session mode using Kubernetes. ( on Flink 1.14.3) We want to migrate towards *Flink Operator* + Application mode deployment setup. ( still continue using Flink 1.14.3) In the current setup, we upload a jar once and t