Hi, That's correct. One app one web UI. Open 4041 and you'll see the other app.
Jacek On 9 Sep 2016 11:53 a.m., "Vladimir Tretyakov" < vladimir.tretya...@sematext.com> wrote: > Hello again. > > I am trying to play with Spark version "2.11-2.0.0". > > Problem that REST API and UI shows me different things. > > I've stared 2 applications from "examples set": opened 2 consoles and run > following command in each: > > *./bin/spark-submit --class org.apache.spark.examples.SparkPi --master > spark://wawanawna:7077 --executor-memory 2G --total-executor-cores 30 > examples/jars/spark-examples_2.11-2.0.0.jar 10000* > > Request to API endpoint: > > http://localhost:4040/api/v1/applications > > returned me following JSON: > > [ { > "id" : "app-20160909184529-0016", > "name" : "Spark Pi", > "attempts" : [ { > "startTime" : "2016-09-09T15:45:25.047GMT", > "endTime" : "1969-12-31T23:59:59.999GMT", > "lastUpdated" : "2016-09-09T15:45:25.047GMT", > "duration" : 0, > "sparkUser" : "", > "completed" : false, > "startTimeEpoch" : 1473435925047, > "endTimeEpoch" : -1, > "lastUpdatedEpoch" : 1473435925047 > } ] > } ] > > so response contains information only about 1 application. But in reality > I've started 2 applications and Spark UI shows me 2 RUNNING application > (please see screenshot). Does anybody maybe know answer why API and UI > shows different things? > > > Best regards, Vladimir. > > > On Tue, Aug 30, 2016 at 3:52 PM, Vijay Kiran <m...@vijaykiran.com> wrote: > >> Hi Otis, >> >> Did you check the REST API as documented in >> http://spark.apache.org/docs/latest/monitoring.html >> >> Regards, >> Vijay >> >> > On 30 Aug 2016, at 14:43, Otis Gospodnetić <otis.gospodne...@gmail.com> >> wrote: >> > >> > Hi Mich and Vijay, >> > >> > Thanks! I forgot to include an important bit - I'm looking for a >> programmatic way to get Spark metrics when running Spark under YARN - so >> JMX or API of some kind. >> > >> > Thanks, >> > Otis >> > -- >> > Monitoring - Log Management - Alerting - Anomaly Detection >> > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ >> > >> > >> > On Tue, Aug 30, 2016 at 6:59 AM, Mich Talebzadeh < >> mich.talebza...@gmail.com> wrote: >> > Spark UI regardless of deployment mode Standalone, yarn etc runs on >> port 4040 by default that can be accessed directly >> > >> > Otherwise one can specify a specific port with --conf >> "spark.ui.port=55555" for example 55555 >> > >> > HTH >> > >> > Dr Mich Talebzadeh >> > >> > LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJ >> d6zP6AcPCCdOABUrV8Pw >> > >> > http://talebzadehmich.wordpress.com >> > >> > Disclaimer: Use it at your own risk. Any and all responsibility for any >> loss, damage or destruction of data or any other property which may arise >> from relying on this email's technical content is explicitly disclaimed. >> The author will in no case be liable for any monetary damages arising from >> such loss, damage or destruction. >> > >> > >> > On 30 August 2016 at 11:48, Vijay Kiran <m...@vijaykiran.com> wrote: >> > >> > From Yarm RM UI, find the spark application Id, and in the application >> details, you can click on the “Tracking URL” which should give you the >> Spark UI. >> > >> > ./Vijay >> > >> > > On 30 Aug 2016, at 07:53, Otis Gospodnetić < >> otis.gospodne...@gmail.com> wrote: >> > > >> > > Hi, >> > > >> > > When Spark is run on top of YARN, where/how can one get Spark metrics? >> > > >> > > Thanks, >> > > Otis >> > > -- >> > > Monitoring - Log Management - Alerting - Anomaly Detection >> > > Solr & Elasticsearch Consulting Support Training - >> http://sematext.com/ >> > > >> > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> > >> > >> > >> >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> > > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >
