ahshahid commented on code in PR #50033:
URL: https://github.com/apache/spark/pull/50033#discussion_r2001966138
##########
resource-managers/yarn/pom.xml:
##########
@@ -37,6 +37,11 @@
<artifactId>spark-core_${scala.binary.version}</artifactId>
<version>${project.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.spark</groupId>
+ <artifactId>spark-sql_${scala.binary.version}</artifactId>
+ <version>${project.version}</version>
+ </dependency>
Review Comment:
@attilapiros I doubt its possible to test the bug using RDD's only, because
unless there are bigger changes in the code spark layer like RDD, Partitioner
interface etc, there is no way to tell an RDD , that the Partitioner its using
is inDeterminate. If an explict Partitioner is provided , than it needs
augmentation to tell whether the nature of partitioner is inDeterminate or not.
For SQL its possible to determine implictly if the partitioning logic is
going to be inDeterminate because of metadata info available with Expression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
