[
https://issues.apache.org/jira/browse/SPARK-59618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59618:
-----------------------------------
Labels: pull-request-available (was: )
> Concurrent F2J LAPACK initialization can hang MultivariateGaussian
> ------------------------------------------------------------------
>
> Key: SPARK-59618
> URL: https://issues.apache.org/jira/browse/SPARK-59618
> Project: Spark
> Issue Type: Bug
> Components: ML
> Affects Versions: 4.2.0
> Reporter: Ruifeng Zheng
> Priority: Major
> Labels: pull-request-available
>
> F2J LAPACK caches machine parameters in unsynchronized mutable static fields.
> During concurrent first use, Dlamc2 publishes first=false before lrmin and
> lrmax are initialized. A racing Dsyev call can consume zero machine limits
> and enter a non-terminating Dlascl scaling loop.
> This can occur when MultivariateGaussian instances evaluate their lazy
> covariance constants concurrently. Repeated thread dumps show one task
> remaining RUNNABLE in Dlascl -> Dsyev -> F2jLAPACK while its peer finishes
> normally.
> The proposed fix warms up the selected LAPACK backend once per executor JVM
> before concurrent eigSym calls. It does not change library dependencies or
> serialize real eigendecompositions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
