Thank you, yes I was thinking about simply running my own thread in UDF and consume some queue something like that. Having some background with DataStreamAPI i was hoping that I can reuse same mechanisms (like Broadcast State Pattern or CoProcessFunction) in Flink SQL. However it seems there is a quite noticeable gap between what you can do with SQL and what you can control comparing to DataStreamAPI.
Regarding UDF being stateless. I assume you mean that UDF does not participate in checkpoint mechanism and I cannot initialize a Flink state in UDF right? I'm wondering why it is not possible. Seems like an "obligatory" feature for Statefull stream processing platform that supports SQL. In a time when there is a huge interest with Flink SQL and everyone is talking about it, thing like state support is not available in SQL UDF is suprissing. Are there any plans, maybe FLIP to change it? Regards, Krzysztof Chmielewski śr., 15 gru 2021 o 02:36 Caizhi Weng <tsreape...@gmail.com> napisał(a): > Hi! > > Currently you can't use broadcast state in Flink SQL UDF because UDFs are > all stateless. > > However you mentioned your use case that you want to control the logic in > UDF with some information. If that is the case, you can just run a thread > in your UDF to read that information and change the behavior of the eval > method accordingly. > > Krzysztof Chmielewski <krzysiek.chmielew...@gmail.com> 于2021年12月15日周三 > 05:47写道： > >> Hi, >> Is there a way to build an UDF [1] for FLink SQL that can be used with >> Broadcast State Pattern [2]? >> >> I have a use case, where I would like to be able to use broadcast control >> stream to change logic in UDF. >> >> Regards, >> Krzysztof Chmielewski >> >> [1] >> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/functions/udfs/#user-defined-functions >> [2] >> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/broadcast_state/#the-broadcast-state-pattern >> >
