j1wonpark opened a new pull request, #7557:
URL: https://github.com/apache/kyuubi/pull/7557
### Why are the changes needed?
DSv2 `TableProvider`s that do not implement `SupportsCatalogOptions` (the
MongoDB
connector, the ClickHouse native connector, etc.), when used directly via
`format(...).load()` / `.save()`, produce a `DataSourceV2Relation` with
neither
`catalog` nor `identifier` — Spark's non-catalog fallback in
`DataSourceV2Utils.loadV2Source` / `DataFrameWriter` passes `(None, None)`,
and the
read-side fallback carries Spark's own TODO note that "Non-catalog paths for
DSV2
are currently not well defined".
For such relations `DataSourceV2RelationTableExtractor` falls back to
`table.name()`, which is connector-defined and usually synthetic (e.g.
`MongoTable()`). The resulting Ranger resource has no `database` element,
and a
resource missing a parent level of the `database → table → column` hierarchy
matches no policy at all — not even `database=* / table=*`
(`RangerDefaultPolicyResourceMatcher.isHierarchyValidForResources`, Ranger
2.6.0).
So the current behavior is an unconditional deny that operators cannot lift
with
any policy, reported in #7230. Meanwhile the v1 path is already skipped: a
`LogicalRelation` without `catalogTable` produces no privilege object.
This PR adds an operator-level opt-out, default `false` so default behavior
is
unchanged:
- `spark.kyuubi.authz.skip.catalogless.v2.relation.enabled` (default
`false`): when
enabled, the extractor returns `None` for a `DataSourceV2Relation` whose
catalog
and identifier are both empty, restoring parity with the v1 path.
- The key is added to `AuthzConfigurationChecker`'s built-in restricted list
(like
`spark.sql.runSQLOnFiles`), so end users cannot flip it with `SET`.
Operators
should also consider listing it in `kyuubi.session.conf.restrict.list`.
Note: whether skipping is safe is deployment-specific — fine when
credentials for
the external system are supplied per query by the end user, risky when the
shared
Spark principal holds ambient credentials. Hence opt-in rather than
unconditional.
Skipped relations produce no Ranger audit events.
### How was this patch tested?
- New `DataSourceV2RelationTableExtractorSuite`:
- default behavior preserved: the synthetic `table.name()` is still
extracted
verbatim when the conf is off;
- the relation is skipped when the conf is enabled;
- relations carrying an identifier are unaffected even with the conf
enabled
(the database fallback from #6544 still applies).
- Extended `AuthzConfigurationCheckerSuite`: `SET` on the new key throws
`AccessControlException`.
### Was this patch authored or co-authored using generative AI tooling?
Assisted-by: Claude Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
