li-boxuan commented on code in PR #49153:
URL: https://github.com/apache/spark/pull/49153#discussion_r2096909096
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryBaseTable.scala:
##########
@@ -491,6 +492,16 @@ abstract class InMemoryBaseTable(
}
}
}
+
+ override def equals(other: Any): Boolean = other match {
+ case imbs: InMemoryBatchScan => this.readSchema == imbs.readSchema &&
Review Comment:
Hi @ahshahid , I am not from Databricks, and unfortunately I am not familar
with Spark code enough to review this PR, but I do encounter some problems with
`equals` overrides in Spark in general, so dropping my point here and see if
you have any idea.
Any reason you don't add
```scala
case imbs: InMemoryBatchScan => this.getClass == imbs.getClass && ...
```
to the equals check? Would this have a negative implication to your use case?
The reason I wanted stricter equals check is because of the project
https://github.com/apache/incubator-gluten. In Gluten, which is a middle layer
between Spark and native engines, (most) operators inherit Spark operators. If
we don't do class equivalence check here, a Spark operator and a Gluten
(native) operator would be regarded as equal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
