j1wonpark opened a new issue, #7746: URL: https://github.com/apache/kyuubi/issues/7746
### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [x] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues. ### Describe the bug With the authz extension enabled, a permanent view over a nondeterministic expression is recomputed for every reference when it is referenced more than once through a CTE. Without the extension Spark computes it once and shares the result through a `ReusedExchange`. ```sql CREATE VIEW v AS SELECT id, reflect('java.util.UUID', 'randomUUID') AS r FROM range(20); WITH c AS (SELECT * FROM v) SELECT a.id, a.r, b.r FROM c a JOIN c b ON a.id = b.id; -- authz enabled: a.r <> b.r on every row; authz disabled: a.r = b.r ``` `InlineCTE.shouldInline` inlines a CTE that is referenced more than once only when `cteDef.deterministic` holds. `PermanentViewMarker` is a `LeafNode`, so `QueryPlan.deterministic` sees no expressions and no children and reports `true` for any view, including one over `rand()`, `uuid()` or `reflect()`. The `Inline CTE` batch runs before the extended optimizer batch that removes the marker, so the wrong answer is what `InlineCTE` sees. For seeded expressions such as `uuid()` and `rand()` the inlined copies happen to agree because both copies share the seed and the partition layout, so the visible symptom is the duplicated work. For anything that draws a fresh value per evaluation the two references disagree. Reproduced on Spark 3.5, 4.0, 4.1 and 4.2 (the marker has been a `LeafNode` since #5780, released in 1.9.0). ### Affects Version(s) master, 1.12.0, 1.11.x, 1.10.x, 1.9.x ### Additional context `SubqueryExpression.deterministic` reads the same flag, so a scalar or IN subquery over such a view is also treated as deterministic under authz (e.g. by `MergeScalarSubqueries`). A PR with a one-line fix (`override lazy val deterministic = child.deterministic`) and tests follows. ### Are you willing to submit PR? - [x] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
