Hi, Thanks for the answers.
@Jacek: To verify if the ui is up, I enter to all the worker nodes of my cluster and run netstat -nltp | grep 4040 with no result. The log of the driver tells me in which server and on which port should the spark ui be up, but it isn't. @Mich: I've tried to specify spark.ui.port=nnn but I only manage to change the log, reporting that the driver should be in another port. The ui has no problem to start in that port (4040) when I run my application in client mode. Could there be a network issue making the ui to fail silently? I've read some of the code regarding those parts of the driver log, but couldn't find anything weird. 2016-07-29 19:45 GMT-03:00 Mich Talebzadeh <mich.talebza...@gmail.com>: > why chance it. Best to explicitly specify in spark-submit (or whatever) > which port to listen to > > --conf "spark.ui.port=nnn" > > and see if it works > > HTH > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 29 July 2016 at 23:37, Jacek Laskowski <ja...@japila.pl> wrote: > >> Hi, >> >> I'm curious about "For some reason, sometimes the SparkUI does not >> appear to be bound on port 4040 (or any other) but the application >> runs perfectly and finishes giving the expected answer." How do you >> check that web UI listens to the port 4040? >> >> Pozdrawiam, >> Jacek Laskowski >> ---- >> https://medium.com/@jaceklaskowski/ >> Mastering Apache Spark http://bit.ly/mastering-apache-spark >> Follow me at https://twitter.com/jaceklaskowski >> >> >> On Thu, Jul 28, 2016 at 11:37 PM, Maximiliano Patricio Méndez >> <mmen...@despegar.com> wrote: >> > Hi, >> > >> > I'm having some trouble trying to submit an application to my spark >> cluster. >> > For some reason, sometimes the SparkUI does not appear to be bound on >> port >> > 4040 (or any other) but the application runs perfectly and finishes >> giving >> > the expected answer. >> > >> > And don't know why, but if I restart all the workers at once sometimes >> it >> > begins to work and sometimes it doesn't. >> > >> > In the driver logs, when it fails to start the SparkUI I see some these >> > lines: >> > 16/07/28 16:13:37 INFO Utils: Successfully started service 'SparkUI' on >> port >> > 4040. >> > 16/07/28 16:13:37 INFO SparkUI: Started SparkUI at >> http://hostname-00:4040 >> > >> > but nothing running in those ports. >> > >> > I'm attaching the full driver log in which I've activated jetty logs on >> > DEBUG but couldn't find anything. >> > >> > The only properties that I'm not leaving at default at the >> configuration is >> > the SPARK_PUBLIC_DNS=$(hostname), SPARK_WORKER_CORES and >> SPARK_WORKER_MEMORY >> > >> > Have anyone faced something similar? >> > >> > Thanks >> > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> >
