Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-06-24 Thread Alexey Leonov-Vendrovskiy
Hey all, starting a vote in 24 hours. Unless there are more points to discuss. Thanks, Alexey On Mon, Jun 17, 2024 at 8:28 AM Alexey Leonov-Vendrovskiy < vendrov...@gmail.com> wrote: > Done. There was some accidental removal during editing. > > Thanks, > Alexey > > On Mon, Jun 17, 2024 at 5:23 A

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-06-17 Thread Alexey Leonov-Vendrovskiy
Done. There was some accidental removal during editing. Thanks, Alexey On Mon, Jun 17, 2024 at 5:23 AM Jim Hughes wrote: > Hi Alexey, > > Looks reasonable. As a note, I think this sentence is missing some words: > > "Due to batch queries nature, we do not expect the jobs to run forever > (days

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-06-17 Thread Jim Hughes
Hi Alexey, Looks reasonable. As a note, I think this sentence is missing some words: "Due to batch queries nature, we do not expect the jobs to run forever (days, months, years) – unlike from . " Can you update it? Cheers, Jim On Mon, Jun 17, 2024 at 3:16 AM Alexey Leonov-Vendrovskiy < vendr

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-06-17 Thread Alexey Leonov-Vendrovskiy
Thanks Timo and Jim! Added a few sentences to the FLIP to cover your points. -Alexey On Mon, Jun 10, 2024 at 11:23 PM Timo Walther wrote: > Hi Alexey, > > thanks for proposing this FLIP. It is a nice continuation of the vision > we had for CompiledPlan when writing and implementing FLIP-190. The

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-06-10 Thread Timo Walther
Hi Alexey, thanks for proposing this FLIP. It is a nice continuation of the vision we had for CompiledPlan when writing and implementing FLIP-190. The whole stack is prepared for serializing BatchExecNodes as well so it shouldn't be too hard to make this a reality. > I think the FLIP should

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-06-07 Thread Jim Hughes
Hi Alexey, Responses inline below: On Mon, May 13, 2024 at 7:18 PM Alexey Leonov-Vendrovskiy < vendrov...@gmail.com> wrote: > Thanks Jim. > > > 1. For the testing, I'd call the tests "execution" tests rather than > > "restore" tests. For streaming execution, restore tests have the > compiled >

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-06-07 Thread Fabian Paul
Thanks, Alexey, for the proposal. I think this is a nice addition that finally fixes the gap in the CompiledPlan. +1 Best, Fabian On Tue, May 14, 2024 at 1:19 AM Alexey Leonov-Vendrovskiy < vendrov...@gmail.com> wrote: > Thanks Jim. > > > > > 1. For the testing, I'd call the tests "execution" te

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-13 Thread Alexey Leonov-Vendrovskiy
Thanks Jim. > 1. For the testing, I'd call the tests "execution" tests rather than > "restore" tests. For streaming execution, restore tests have the compiled > plan and intermediate state; the tests verify that those can work together > and continue processing. Agree that we don't need to st

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-13 Thread Jim Hughes
Hi Alexey, After some thought, I have a question about deprecations: 3. Can we remove old versions of batch operators eventually? Or do we need to keep them forever like we would for streaming operators? Cheers, Jim On Thu, May 9, 2024 at 11:29 AM Jim Hughes wrote: > Hi Alexey, > > Overall

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-09 Thread Jim Hughes
Hi Alexey, Overall, the FLIP looks good and makes sense to me. 1. For the testing, I'd call the tests "execution" tests rather than "restore" tests. For streaming execution, restore tests have the compiled plan and intermediate state; the tests verify that those can work together and continue pr

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-07 Thread Alexey Leonov-Vendrovskiy
Hey Paul, Yes, no interchangeability. Just a wire-through for more uniformity. Thanks, Alexey On Tue, May 7, 2024 at 2:10 AM Paul Lam wrote: > Hi Alexey, > > Thanks a lot for bringing up the discussion. I’m big +1 for the FLIP. > > I suppose the goal doesn’t involve the interchangeability of j

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-07 Thread Paul Lam
Hi Alexey, Thanks a lot for bringing up the discussion. I’m big +1 for the FLIP. I suppose the goal doesn’t involve the interchangeability of json plans between batch mode and streaming mode, right? In other words, a json plan compiled in a batch program can’t be run in streaming mode without a

[DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-06 Thread Alexey Leonov-Vendrovskiy
Hi everyone, PTAL at the proposed FLIP-456: CompiledPlan support for Batch Execution Mode. It is pretty self-describing. Any thoughts are welcome! Thanks, Alexey [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-456%3A+CompiledPlan+support+for+Batch+Execution+Mode .