Tengfei Huang created SPARK-59242:
-------------------------------------
Summary: NPE in SQLExecution.withNewExecutionId masks the real
query failure when the SparkContext is stopped concurrently
Key: SPARK-59242
URL: https://issues.apache.org/jira/browse/SPARK-59242
Project: Spark
Issue Type: Task
Components: SQL
Affects Versions: 4.2.0
Reporter: Tengfei Huang
`SQLExecution.withNewExecutionId` dereferences sc.dagScheduler inside its
finally block. SparkContext.stop() sets _dagScheduler to null before it stops
the listener bus, so a query that finishes while the context is being torn down
reaches that line with a null receiver and throws a `NullPointerException`.
Because the NPE is thrown from a finally, it replaces the query's original
exception rather than being suppressed, and it skips every remaining cleanup
step in the block.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
