Re: UDF and Broadcast State Pattern

2021-12-15 Thread Krzysztof Chmielewski
Thank you very much for the clarification Seth. Best Regards Krzysztof Chmielewski śr., 15 gru 2021, 16:12 użytkownik Seth Wiesman napisał: > Hi Krzysztof, > > There is a difference in semantics here between yourself and Caizhi. SQL > UDFs can be used statefully - see AggregateFunction and > T

Re: UDF and Broadcast State Pattern

2021-12-15 Thread Seth Wiesman
Hi Krzysztof, There is a difference in semantics here between yourself and Caizhi. SQL UDFs can be used statefully - see AggregateFunction and TableAggregateFunction for examples. You even have access to ListView and MapView which are backed by ListState and MapState accordingly. These functions c

Re: UDF and Broadcast State Pattern

2021-12-15 Thread Krzysztof Chmielewski
Thank you, yes I was thinking about simply running my own thread in UDF and consume some queue something like that. Having some background with DataStreamAPI i was hoping that I can reuse same mechanisms (like Broadcast State Pattern or CoProcessFunction) in Flink SQL. However it seems there is a q

Re: UDF and Broadcast State Pattern

2021-12-14 Thread Caizhi Weng
Hi! Currently you can't use broadcast state in Flink SQL UDF because UDFs are all stateless. However you mentioned your use case that you want to control the logic in UDF with some information. If that is the case, you can just run a thread in your UDF to read that information and change the beha

UDF and Broadcast State Pattern

2021-12-14 Thread Krzysztof Chmielewski
Hi, Is there a way to build an UDF [1] for FLink SQL that can be used with Broadcast State Pattern [2]? I have a use case, where I would like to be able to use broadcast control stream to change logic in UDF. Regards, Krzysztof Chmielewski [1] https://nightlies.apache.org/flink/flink-docs-master