Re: Process table function scan operation in batch mode is not supported

2025-04-05 Thread Timo Walther
Hi Gabor, great that you already try out PTFs. I'm in the process of writing documentation for it. Including a list of limitations. Please note that PTF won't be support in batch mode in the first phase. For stateful PTFs we would need to use a batch state backend and also other code paths a

Re: Process table function scan operation in batch mode is not supported

2025-03-28 Thread Shengkai Fang
Hi, Timo. As far as I know, table function should work with `lateral` keyword, it means users can not use the following statement to read the results? ``` SELECT * FROM read_metadata('') ``` Best, Shengkai Gabor Somogyi 于2025年3月28日周五 16:18写道: > Hi Timo, > > Thanks for the help! > Your comm

Re: Process table function scan operation in batch mode is not supported

2025-03-28 Thread Gabor Somogyi
Hi Timo, Thanks for the help! Your comment is fair and shows the proper direction in the mentioned FLIP. BR, G On Fri, Mar 28, 2025 at 8:36 AM Gyula Fóra wrote: > That's a very good point, I let Gabor and Shengkai follow up on that > suggestion :) > > Gyula > > On Fri, Mar 28, 2025 at 8:30 AM

Re: Process table function scan operation in batch mode is not supported

2025-03-28 Thread Gyula Fóra
That's a very good point, I let Gabor and Shengkai follow up on that suggestion :) Gyula On Fri, Mar 28, 2025 at 8:30 AM Timo Walther wrote: > Hi Gyula, > > if I understand the discussion correctly, you want to use a PTF without > table arguments to return a table (read from savepoint metadata)

Re: Process table function scan operation in batch mode is not supported

2025-03-28 Thread Gyula Fóra
Hi Timo! Thanks for the answers. Just to give some context here is this thread: https://lists.apache.org/thread/08jwrocqyk1q82lnfdldhnyb79m496lp We were considering a PTF like state_metadata("checkpointpath") to create a table with the available state metadata instead of creating a custom connec

Re: Process table function scan operation in batch mode is not supported

2025-03-28 Thread Timo Walther
Hi Gyula, if I understand the discussion correctly, you want to use a PTF without table arguments to return a table (read from savepoint metadata)? If this is the case, you don't need a PTF for it. A regular table function can also do the job. IIRC we support TVF with constant args. Cheers,

Re: Process table function scan operation in batch mode is not supported

2025-03-27 Thread Shengkai Fang
I think it is by design. You can read the FLIP, it says: *Time Semantics*: - PTFs support event-time semantics only. - Processing-time doesn’t go well with batch mode and thus a unified API should built on event-time. The proposed onWatermark timers allow for making processing

Process table function scan operation in batch mode is not supported

2025-03-27 Thread Gabor Somogyi
Hi All, Seems like the process table function scan operation is not supported in batch mode. Steps to repro [1] which gives the following exception: Caused by: org.apache.flink.table.api.TableException: Unsupported function for scan:PROCESS_TABLE Is this something which is planned? [1] https://