http://spark.apache.org/docs/latest/monitoring.html#spark-configuration-options
says
"Note that this information is only available for the duration of the
application by default. To view the web UI after the fact, set
spark.eventLog.enabled to true before starting the application. This
configures
Thanks Zahid, Yes I am using history server to see previous UIs.
However, my question still remains on viewing old thread dumps, as I
cannot see them on the old completed spark UIs, only when spark context is
running.
On Wed, Apr 8, 2020 at 4:01 PM Zahid Rahman wrote:
> Spark UI is only availa
Spark UI is only available while SparkContext is running.
However You can get to the Spark UI after your application completes or
crashes.
To do this Spark includes a tool called the Spark History Server that
allows you to reconstruct the Spark UI.
You can find up to date information on how to
Hi all,
As stated in title, currently when I view the spark UI of a completed spark
job, I see there are thread dump links in the executor tab, but clicking on
them does nothing. Is it possible to see the thread dumps somehow even if
the job finishes? On spark 2.4.5.
Thanks.
--
Cheers,
Ruijing L
