jarredhj0214 opened a new pull request, #13119: URL: https://github.com/apache/gravitino/pull/13119
### What changes were proposed in this pull request? This PR adds a job-level switch for the Gravitino Spark connector: ```properties spark.sql.gravitino.enabled=false ``` When this property is set to `false`, `GravitinoDriverPlugin.init()` returns before reading or validating Gravitino URI and metalake settings. This skips Gravitino client creation, server access, catalog loading, Spark catalog registration, and Gravitino SQL extension injection while still allowing Spark to load the configured plugin class. ### Why are the changes needed? Some platforms inject the Gravitino Spark plugin globally, but not every existing Spark job or third-party Spark component is compatible with the connector. TiSpark is one example, but the general problem is that Gravitino cannot guarantee compatibility with every Spark workload during incremental rollout. This switch lets individual jobs opt out of Gravitino initialization without requiring the platform to remove the global `spark.plugins` configuration. It helps avoid blocking existing jobs while Gravitino is adopted gradually. Fix: #13117 ### Does this PR introduce _any_ user-facing change? Yes. It adds a new Spark configuration: ```properties spark.sql.gravitino.enabled ``` The default value is `true`, so existing behavior is unchanged. Setting it to `false` disables Gravitino Spark connector initialization for that Spark application. ### How was this patch tested? Added unit tests in `TestGravitinoDriverPlugin` to cover: - Default behavior remains enabled and still requires Gravitino URI - Disabled mode does not require Gravitino URI or metalake - Disabled mode does not access Spark user information - Disabled mode does not register Spark catalogs - Disabled mode does not inject Gravitino SQL extensions, even when Iceberg/Paimon support flags are set Ran: ```shell JAVA_HOME=/opt/homebrew/opt/openjdk@17 ./gradlew :spark-connector:spotlessApply JAVA_HOME=/opt/homebrew/opt/openjdk@17 ./gradlew :spark-connector:spotlessApply :spark-connector:spark-3.5:test --tests org.apache.gravitino.spark.connector.plugin.TestGravitinoDriverPlugin -PskipITs git diff --check ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
