Re: Dynamic configuration via broadcast state

2021-04-09 Thread Arvid Heise
Hi Vishal, what you are trying to achieve is quite common and has its own documentation [1]. Currently, there is no way to hold back elements of the non-broadcast side (your question 2 in OP), so you have to save them until configuration arrives. If you have several configurable operators, you co

Re: Dynamic configuration via broadcast state

2021-04-06 Thread vishalovercome
I researched a bit more and another suggested solution is to build a custom source function that somehow waits for each operator to load it's configuration which is infact set in the open method of the source itself. I'm not sure if that's a good idea as that just exposes entire job configuration t