[
https://issues.apache.org/jira/browse/SPARK-57313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-57313.
---------------------------------
Fix Version/s: 4.2.0
Resolution: Fixed
Issue resolved by pull request 56363
[https://github.com/apache/spark/pull/56363]
> SampleExec does not update numOutputRows metric when whole-stage codegen is
> disabled
> ------------------------------------------------------------------------------------
>
> Key: SPARK-57313
> URL: https://issues.apache.org/jira/browse/SPARK-57313
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.2.0
>
> Attachments: image-2026-06-07-23-09-27-749.png,
> image-2026-06-07-23-09-51-934.png, image-2026-06-07-23-10-47-398.png
>
>
> `SampleExec` declares a `numOutputRows` SQL metric. Its whole-stage-codegen
> path (`doConsume`) increments it per emitted row, but the interpreted
> doExecute returns the sampled RDD without ever updating the metric.
> As a result, when whole-stage codegen is off
> (spark.sql.codegen.wholeStage=false), the Sample / TABLESAMPLE operator
> reports number of output rows: 0 in the SQL UI, while the codegen path
> reports the correct count.
>
> Repro:
> {code:java}
> spark.conf.set("spark.sql.codegen.wholeStage", "false")
> val df = spark.range(0, 1000, 1, 1).sample(withReplacement = false,
> fraction = 0.5, seed = 1)
> df.collect()
> // Sample node in the SQL tab shows "number of output rows: 0" (expected
> ~518) {code}
> wscg = true
> !image-2026-06-07-23-09-51-934.png|width=669,height=375!
>
> wscg=false
> !image-2026-06-07-23-10-47-398.png|width=674,height=333!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
