Hello Jacek, thx a lot, it works. Is there a way how to get list of running applications from REST API? Or I have to try connect 4040 4041... 40xx ports and check if ports answer something?
Best regards, Vladimir. On Sat, Sep 10, 2016 at 6:00 AM, Jacek Laskowski <ja...@japila.pl> wrote: > Hi, > > That's correct. One app one web UI. Open 4041 and you'll see the other > app. > > Jacek > > On 9 Sep 2016 11:53 a.m., "Vladimir Tretyakov" < > vladimir.tretya...@sematext.com> wrote: > >> Hello again. >> >> I am trying to play with Spark version "2.11-2.0.0". >> >> Problem that REST API and UI shows me different things. >> >> I've stared 2 applications from "examples set": opened 2 consoles and run >> following command in each: >> >> *./bin/spark-submit --class org.apache.spark.examples.SparkPi >> --master spark://wawanawna:7077 --executor-memory 2G >> --total-executor-cores 30 examples/jars/spark-examples_2.11-2.0.0.jar >> 10000* >> >> Request to API endpoint: >> >> http://localhost:4040/api/v1/applications >> >> returned me following JSON: >> >> [ { >> "id" : "app-20160909184529-0016", >> "name" : "Spark Pi", >> "attempts" : [ { >> "startTime" : "2016-09-09T15:45:25.047GMT", >> "endTime" : "1969-12-31T23:59:59.999GMT", >> "lastUpdated" : "2016-09-09T15:45:25.047GMT", >> "duration" : 0, >> "sparkUser" : "", >> "completed" : false, >> "startTimeEpoch" : 1473435925047, >> "endTimeEpoch" : -1, >> "lastUpdatedEpoch" : 1473435925047 >> } ] >> } ] >> >> so response contains information only about 1 application. But in reality >> I've started 2 applications and Spark UI shows me 2 RUNNING application >> (please see screenshot). Does anybody maybe know answer why API and UI >> shows different things? >> >> >> Best regards, Vladimir. >> >> >> On Tue, Aug 30, 2016 at 3:52 PM, Vijay Kiran <m...@vijaykiran.com> wrote: >> >>> Hi Otis, >>> >>> Did you check the REST API as documented in >>> http://spark.apache.org/docs/latest/monitoring.html >>> >>> Regards, >>> Vijay >>> >>> > On 30 Aug 2016, at 14:43, Otis Gospodnetić <otis.gospodne...@gmail.com> >>> wrote: >>> > >>> > Hi Mich and Vijay, >>> > >>> > Thanks! I forgot to include an important bit - I'm looking for a >>> programmatic way to get Spark metrics when running Spark under YARN - so >>> JMX or API of some kind. >>> > >>> > Thanks, >>> > Otis >>> > -- >>> > Monitoring - Log Management - Alerting - Anomaly Detection >>> > Solr & Elasticsearch Consulting Support Training - >>> http://sematext.com/ >>> > >>> > >>> > On Tue, Aug 30, 2016 at 6:59 AM, Mich Talebzadeh < >>> mich.talebza...@gmail.com> wrote: >>> > Spark UI regardless of deployment mode Standalone, yarn etc runs on >>> port 4040 by default that can be accessed directly >>> > >>> > Otherwise one can specify a specific port with --conf >>> "spark.ui.port=55555" for example 55555 >>> > >>> > HTH >>> > >>> > Dr Mich Talebzadeh >>> > >>> > LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJ >>> d6zP6AcPCCdOABUrV8Pw >>> > >>> > http://talebzadehmich.wordpress.com >>> > >>> > Disclaimer: Use it at your own risk. Any and all responsibility for >>> any loss, damage or destruction of data or any other property which may >>> arise from relying on this email's technical content is explicitly >>> disclaimed. The author will in no case be liable for any monetary damages >>> arising from such loss, damage or destruction. >>> > >>> > >>> > On 30 August 2016 at 11:48, Vijay Kiran <m...@vijaykiran.com> wrote: >>> > >>> > From Yarm RM UI, find the spark application Id, and in the application >>> details, you can click on the “Tracking URL” which should give you the >>> Spark UI. >>> > >>> > ./Vijay >>> > >>> > > On 30 Aug 2016, at 07:53, Otis Gospodnetić < >>> otis.gospodne...@gmail.com> wrote: >>> > > >>> > > Hi, >>> > > >>> > > When Spark is run on top of YARN, where/how can one get Spark >>> metrics? >>> > > >>> > > Thanks, >>> > > Otis >>> > > -- >>> > > Monitoring - Log Management - Alerting - Anomaly Detection >>> > > Solr & Elasticsearch Consulting Support Training - >>> http://sematext.com/ >>> > > >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>> > >>> > >>> > >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >
