Haiyang Sun created SPARK-57371:
-----------------------------------
Summary: Fix thread-unsafe Scala reflection race in
ArrowDeserializers.resolveCompanion
Key: SPARK-57371
URL: https://issues.apache.org/jira/browse/SPARK-57371
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 4.3.0
Reporter: Haiyang Sun
Fixes a thread-safety race condition in ArrowDeserializers.resolveCompanion()
that causes `ScalaReflectionException: <none> is not a module` when used
concurrently.
Scala runtime reflection is not thread-safe (scala/bug#6240, #10431). The
resolveCompanion method calls `mirror.classSymbol(...).companion.asModule`
without synchronization. Under concurrent access - companion can race and
return NoSymbol, causing `.asModule` to throw. The same issue affects
`ScalaEnumEncoder` deserialization.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
