I have no other advice for this. Does the situation improve after parameter
configuration?
Jason Jun 于2022年8月5日周五 06:55写道：
> Hi Qian,
>
> Thanks for your feedback. We're using spark ver 3.1.2, these are set :
>
> spark.ui.retainedJobs 10
> spark.ui.retainedStages 10
> spark.ui.retainedTasks 100
Hi Jason
LiveUI initializes ElementTrackingStore with InMemoryStore, so it has OOM risk.
/**
* Create an in-memory store for a live application.
*/
def createLiveStore(
conf: SparkConf,
appStatusSource: Option[AppStatusSource] = None): AppStatusStore = {
val store = new ElementTracking
He there,
We have spark driver running 24x7, and we are continiously getting OOM in
spark driver every 10 days.
I found org.apache.spark.status.ElementTrackingStore keep 85% of heap usage
after analyzing heap dump like this image:
[image: image.png]
i found these parameter would be the root cause
