Re: [DISCUSS] addition of Flink Script Operators

2023-08-07 Thread Kevin Tseng
Hi Jing, Thanks for the input, that's exactly how I have imagined the standard deployment process. With regards to the Exactly-Once guarantee as per my understanding: Current Exactly-Once semantics are performed via Two Phase Commit that is done at both Source and Sink (plus any checkpoint ena

Re: [DISCUSS] addition of Flink Script Operators

2023-08-04 Thread Jing Ge
Hi Kevin, Thanks for your proposal. It is quite interesting. We had a similar solution back to the Apache Storm era many years ago. The job deployment with operators itself could be considered as physical distributed task containers. There will be a management tool, like a control plane, to deploy