rishvin opened a new issue, #2214:
URL: https://github.com/apache/datafusion-comet/issues/2214
### Describe the bug
I invoked the following in the spark-shell, version `3.5.6`.
```
$SPARK_HOME/bin/spark-shell
$COMET/spark/target/comet-spark-spark3.5_2.12-0.10.0-SNAPSHOT --conf
spark.plugins=org.apache.spark.CometPlugin --conf spark.comet.enabled=true
--conf spark.comet.exec.enabled=true
```
```
import org.apache.spark.sql.types._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Row
val schema = StructType(Seq(StructField("id", IntegerType, nullable =
false), StructField("value", IntegerType, nullable = false)))
val data = Seq(Row(1, 10), Row(2, 20), Row(3, 10), Row(4, 30), Row(5, 20),
Row(6, 10))
val df = spark.createDataFrame(spark.sparkContext.parallelize(data), schema)
val out = "groupby"
df.write.mode("overwrite").parquet(out)
val parquetDF = spark.read.parquet(out)
val grouped = parquetDF.groupBy("id").count()
grouped.explain()
```
And saw that the explain output has `CometHashAggregate` prefixed with `!`.
```
== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=false
+- HashAggregate(keys=[id#137], functions=[count(1)])
+- Exchange hashpartitioning(id#137, 4), ENSURE_REQUIREMENTS,
[plan_id=420]
+- !CometHashAggregate [id#137], Partial, [id#137], [partial_count(1)]
+- CometNativeScan parquet [id#137] Batched: true, DataFilters: [],
Format: CometParquet, Location: InMemoryFileIndex(1
paths)[file:/home/testing/groupby], PartitionFilters: [], PushedFilters: [],
ReadSchema: struct<id:int>
```
**There was no correctness issues and no failures.**
The spark codebase
[here](https://github.com/apache/spark/blob/303c18c74664f161b9b969ac343784c088b47593/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala#L425)
says - they use `"!" to indicate an invalid plan, and "'" to indicate an
unresolved plan`, although, I am not fully sure, if `!` is getting added from
the same place in the code.
I just saw the comment in the spark-code and filed this issue to bring to
notice.
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
