Fabian Hueske created FLINK-40079:
-------------------------------------
Summary: Enforce disableSystemArguments consistently for all PTF
translation paths
Key: FLINK-40079
URL: https://issues.apache.org/jira/browse/FLINK-40079
Project: Flink
Issue Type: Sub-task
Components: Table SQL / Planner
Reporter: Fabian Hueske
{{BuiltInFunctionDefinition.Builder.disableSystemArguments(true)}} on a PTF
definition is intended to mean "forbid the implicit system arguments
(on_time/uid)" for this function.
That semantics is not really implemented anywhere, and behavior differs by
translation path:
* {{FlinkCalciteSqlValidator}} recognizes on_time/uid for every PTF and
ignores the flag, so validation always accepts them.
* The only place the flag is consulted is
{{{}StreamPhysicalProcessTableFunction.validateAllowSystemArgs{}}}, which
throws unconditionally when the flag is set — rejecting the entire call,
regardless of whether a system argument was supplied.
Resulting inconsistency:
* Generic PTF physical rule (regular path): the whole call is hard-rejected,
even with no system argument (e.g. {{{}SELECT * FROM SNAPSHOT(input => TABLE
t){}}}), with a misleading "not supported for user-defined PTF" message.
* Dedicated optimizer rule (bypasses that physical rule): the flag is never
checked, so on_time/uid are silently accepted and ignored. Known cases:
SNAPSHOT in a LATERAL context
({{{}LogicalCorrelateToJoinFromLateralSnapshotRule{}}}), likely ML_PREDICT
({{{}StreamPhysicalMLPredictTableFunctionRule{}}}).
Proposed fix: implement the intended semantics at SQL validation so it holds
for every path .When a PTF sets {{{}disableSystemArguments(true){}}}, reject
any supplied on_time/uid with a clear ValidationException, independent of later
translation.
Open decision for the generic (regular) path, given its runtime currently
assumes system args exist:
* (a) keep rejecting flag-set PTFs there, but with a precise message
explaining the function's required usage; or
* (b) allow them (call permitted, system args rejected) if the runtime can
support it.
Acceptance:
* Supplying on_time/uid to a disableSystemArguments(true) PTF fails validation
with a clear message in every context (standalone/generic, LATERAL,
dedicated-rule).
* Generic-path behavior for a flag-set PTF is made consistent per the decision
above (no misleading message; no silent accept).
* Confirm and cover the ML_PREDICT case.
* Add tests ensuring correct rejection of queries using system-args on a
function that forbids them (like SNAPSHOT)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
