Hi Aurélien,
Spark has endpoints to expose the spark application metrics. These endpoints
can be used as a rest API. You can read more about these here:
https://spark.apache.org/docs/3.1.1/monitoring.html#rest-api
Additionally,
If you want to build your own custom metrics, you can explore spark
& Regards,
Akshay Haryani
From: Aurélien Mazoyer
Date: Monday, September 6, 2021 at 5:47 AM
To: Haryani, Akshay
Cc: user@spark.apache.org
Subject: Re: Get application metric from Spark job
Hi Akshay,
Thank you for your reply. Sounds like a good idea, but I unfortunately have a
2.6 cluster. Do
