cnauroth opened a new pull request, #50156: URL: https://github.com/apache/spark/pull/50156
### What changes were proposed in this pull request? In the `scalastyle` helper script, activate profiles docker-integration-tests and kubernetes-integration-tests in the default value of `SPARK_PROFILES` instead of directly on the `sbt` command line. ### Why are the changes needed? The `scalastyle` script supports customization of which profiles to activate and check for style violations by passing command-line arguments that fill the `SPARK_PROFILES` environment variable. However, the docker-integration-tests and kubernetes-integration-tests profiles are always activated directly during the `sbt` invocation. This may waste time when engineers want to check for style violations on work in progress that doesn't change any code under these profiles. ### Does this PR introduce _any_ user-facing change? No. This change only impacts developer tooling. The default behavior of the script when no arguments are passed remains the same. ### How was this patch tested? In my local repo (not included in this pull request), I introduced style violations in docker-integration-tests and kubernetes-integration-tests. I then ran the script with no arguments, and it flagged the style violations: ``` dev/scalastyle Scalastyle checks failed at following occurrences: [error] /usr/local/google/home/cnauroth/git/apache/spark/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/Utils.scala:20:0: java.io. is in wrong order relative to java.nio.file.. [error] /usr/local/google/home/cnauroth/git/apache/spark/connector/docker-integration-tests/src/test/scala/org/apache/spark/util/DockerUtils.scala:23:0: scala.jdk.CollectionConverters._ is in wrong order relative to scala.sys.process._. [error] Total time: 66 s (01:06), completed Mar 4, 2025, 1:16:42 AM ``` I repeated this with command-line arguments specifying just the yarn profile, but it still flagged the violations in docker-integration-tests and kubernetes-integration-tests: ``` dev/scalastyle -Pyarn Scalastyle checks failed at following occurrences: [error] /usr/local/google/home/cnauroth/git/apache/spark/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/Utils.scala:20:0: java.io. is in wrong order relative to java.nio.file.. [error] /usr/local/google/home/cnauroth/git/apache/spark/connector/docker-integration-tests/src/test/scala/org/apache/spark/util/DockerUtils.scala:23:0: scala.jdk.CollectionConverters._ is in wrong order relative to scala.sys.process._. [error] Total time: 101 s (01:41), completed Mar 4, 2025, 9:04:29 PM ``` After applying the changes in the script, it still flags the violations in docker-integration-tests and kubernetes-integration-tests by default, but not when I specify only the yarn profile: ``` dev/scalastyle -Pyarn Scalastyle checks passed. ``` ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
