bowenliang123 opened a new pull request, #7596:
URL: https://github.com/apache/kyuubi/pull/7596
### Why are the changes needed?
Spark authz's Paimon catalog e2e suite
(`PaimonCatalogRangerSparkExtensionSuite`) already has 37 SQL-level tests, but
Paimon is missing the test layers Iceberg has:
- A plan-level `*PrivilegesBuilderSuite` that asserts the analyzed
`LogicalPlan` yields the right `PrivilegeObject`s, `OperationType` and
`AccessType` for the Paimon-specific commands. Without it, a regression in the
spec JSON (`table_command_spec.json`) or the extractor would only show up in
the e2e suite as a wrong error message — never as a precise plan-level diff.
- `RowFilteringFor*` and `DataMaskingFor*` suites for Paimon. Paimon users
running with row-filter and data-masking policies get no test coverage today.
This PR closes that gap. It also tightens the existing Paimon e2e suite
(`Producers` → `Procedures`, propagated the `comment = "Paimon"` metadata to
all four `tableDescs` to match the Iceberg/Hudi/Delta style) and adds two
opt-out flags (`supportsTruncateTable`, `supportsOverwritePartitionsDynamic`)
to the shared `V2CommandsPrivilegesSuite` so the inherited generic V2 tests can
be skipped for catalogs whose TRUNCATE / dynamic partition overwrite resolves
to a different logical plan. Default is `true`; only the Paimon subclass flips
them.
No production code is changed.
### How was this patch tested?
Targeted ScalaTest runs on the default Spark 3.5 / Paimon 0.8.2 / Scala 2.12
build:
```
mvn -pl extensions/spark/kyuubi-spark-authz \
-Pspark-3.5,hive-provided,spark-provided,flink-provided -Dtest=none \
-DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.ranger.PaimonCatalogRangerSparkExtensionSuite
test
mvn -pl extensions/spark/kyuubi-spark-authz \
-Pspark-3.5,hive-provided,spark-provided,flink-provided -Dtest=none \
-DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.PaimonCatalogPrivilegesBuilderSuite
test
mvn -pl extensions/spark/kyuubi-spark-authz \
-Pspark-3.5,hive-provided,spark-provided,flink-provided -Dtest=none \
-DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.ranger.rowfiltering.RowFilteringForPaimonSuite
test
mvn -pl extensions/spark/kyuubi-spark-authz \
-Pspark-3.5,hive-provided,spark-provided,flink-provided -Dtest=none \
-DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.ranger.datamasking.DataMaskingForPaimonSuite
test
```
Combined: 137 passed, 0 failed, 18 canceled (intentional, e.g. Paimon `*` is
gated behind the Spark 3.4 `Procedures` check; permanent-view tests cancel
because Paimon's SparkCatalog has no view support).
Regression check on the two existing plan suites that share
`V2CommandsPrivilegesSuite`:
```
mvn -pl extensions/spark/kyuubi-spark-authz \
-Pspark-3.5,hive-provided,spark-provided,flink-provided -Dtest=none \
-DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.IcebergCatalogPrivilegesBuilderSuite
test
mvn -pl extensions/spark/kyuubi-spark-authz \
-Pspark-3.5,hive-provided,spark-provided,flink-provided -Dtest=none \
-DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.V2JdbcTableCatalogPrivilegesBuilderSuite
test
mvn -pl extensions/spark/kyuubi-spark-authz \
-Pspark-3.5,hive-provided,spark-provided,flink-provided -Dtest=none \
-DwildcardSuites=org.apache.kyuubi.plugin.spark.authz.gen.JsonSpecFileGenerator
test
```
Iceberg plan suite: 80 passed, 12 canceled. V2Jdbc plan suite: 83 passed, 10
canceled. Spec generator: 1 passed (`133 specs` regenerated into
`table_command_spec.json`).
Spotless applied across all six Spark profiles (default plus
`-Pspark-3.3/3.4/3.5/4.0/4.1/4.2`).
### Was this patch authored or co-authored using generative AI tooling?
Yes.
Assisted-by: MiniMax-M3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
