andygrove commented on code in PR #4891:
URL: https://github.com/apache/datafusion-comet/pull/4891#discussion_r3562184678
##########
docs/source/user-guide/latest/understanding-comet-plans.md:
##########
@@ -108,6 +109,44 @@ when you want to see all reasons, including ones that
not include the surrounding plan, so it is best for accumulating diagnostics
across many queries.
+### `spark.comet.explainCodegen.enabled`
+
+Disabled by default. When enabled, every Spark expression that Comet routes
+through the JVM codegen dispatcher (Spark's own `doGenCode` running inside a
+Comet kernel — the path used for expressions with no native DataFusion
+implementation, gated by `spark.comet.exec.scalaUDF.codegen.enabled=true`) is
+annotated with `<expr_name>: routes through JVM codegen dispatcher`. The
+annotation rolls up onto the surrounding `CometProject` / `CometFilter` /
+etc. and is rendered inside the `[COMET-INFO: ...]` segment of extended
+explain output (verbose format). The projection stays Comet-native — this is
+informational only, useful for distinguishing "runs natively in DataFusion"
+from "runs Spark's generated code inside a Comet kernel".
+
+Example:
+
+```scala
+spark.conf.set("spark.comet.exec.scalaUDF.codegen.enabled", "true")
+spark.conf.set("spark.comet.explainCodegen.enabled", "true")
+
+val df = spark.sql("SELECT hypot(a, b), levenshtein(s1, s2) FROM t")
+println(new org.apache.comet.ExtendedExplainInfo()
+ .generateExtendedInfo(df.queryExecution.executedPlan))
+```
+
+Output:
+
+```
+CometNativeColumnarToRow
++- CometProject [COMET-INFO: hypot: routes through JVM codegen dispatcher,
levenshtein: routes through JVM codegen dispatcher]
Review Comment:
Is it possible to combine the list of expressions to avoid duplication e.g.
`hypot and Levenshtein route through the JVM codegen dispatcher` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
