mentation for streaming, so you can you profit from testing there too.
>>
>>
>>
>>
>>
>> *From:* Eric Beabes
>> *Sent:* Friday, January 8, 2021 04:23
>> *To:* Luca Canali
>> *Cc:* spark-user
>> *Subject:* Re: Understanding Executors U
streaming, so you can you profit from testing there too.
>
>
>
>
>
> *From:* Eric Beabes
> *Sent:* Friday, January 8, 2021 04:23
> *To:* Luca Canali
> *Cc:* spark-user
> *Subject:* Re: Understanding Executors UI
>
>
>
> So when I see this for 'Storage Memor
proved memory instrumentation and improved
instrumentation for streaming, so you can you profit from testing there too.
From: Eric Beabes
Sent: Friday, January 8, 2021 04:23
To: Luca Canali
Cc: spark-user
Subject: Re: Understanding Executors UI
So when I see this for 'Storage Memory': 3.3TB/
So when I see this for 'Storage Memory': *3.3TB/ 598.5 GB* *- it's telling
me that Spark is using 3.3 TB of memory & 598.5 GB is used for caching
data, correct?* What I am surprised about is that these numbers don't
change at all throughout the day even though the load on the system is low
after 5p
Hi Eric,
A few links, in case they can be useful for your troubleshooting:
The Spark Web UI is documented in Spark 3.x documentation, although you can use
most of it for Spark 2.4 too: https://spark.apache.org/docs/latest/web-ui.html
Spark memory management is documented at
https://spark.apac
