Thanks Nguyen for the link. I installed Super Refresh as ADD on to Chrome. By default the refresh is stop until you set it to x seconds. However, the issue we have is that Spark UI comes with 6+ tabs and you have to repeat the process for each tab.
However, that messes up the things. For example if I choose to refresh "Executors" tab every 2 seconds and decide to refresh "Stages" tab, then there is a race condition whereas you are thrown back to the last refresh page which is not really what one wants. Ideally one wants the Spark UI page identified by host:port to be the driver and every other tab underneath say host:port/Stages to be refreshed once we open that tab and stay there. If I go back to say SQL tab, I like to see that refreshed ever n seconds. I hope this makes sense. Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On 27 August 2016 at 15:01, nguyen duc Tuan <newvalu...@gmail.com> wrote: > The simplest solution that I found: using an browser extension which do > that for you :D. For example, if you are using Chrome, you can use this > extension: https://chrome.google.com/webstore/detail/easy-auto-refresh/ > aabcgdmkeabbnleenpncegpcngjpnjkc/related?hl=en > An other way, but a bit more manually using javascript: start with a > window, you will create a child window with your target url. The parent > window will refresh that child window for you. Due to same-original > pollicy, you should set parent url to the same url as your target url. Try > this in your web console: > wi = window.open("your target url") > var timeInMinis = 2000 > setInterval(function(){ wi.location.reload();}, timeInMinis) > Hope this help. > > 2016-08-27 20:17 GMT+07:00 Mich Talebzadeh <mich.talebza...@gmail.com>: > >> Hi All, >> >> GitHub project SparkUIDashboard created here >> <https://github.com/search?utf8=%E2%9C%93&q=sparkuidashboard&ref=simplesearch> >> >> >> >> >> [image: Inline images 2] >> Let use put the show on the road :) >> >> Cheers >> >> >> Dr Mich Talebzadeh >> >> >> >> LinkedIn * >> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >> On 27 August 2016 at 12:53, Sivakumaran S <siva.kuma...@icloud.com> >> wrote: >> >>> Hi Mich, >>> >>> Unlikely that we can use Zeppelin for dynamic, real time update >>> visualisation. It makes nice, static visuals. >>> >>> I was thinking more on the lines of http://dashingdemo.herokuap >>> p.com/sample >>> >>> The library is http://dashing.io >>> >>> There are more widgets that can be used https://github.com/Shopif >>> y/dashing/wiki/Additional-Widgets >>> >>> The Spark UI is functional, but I am looking forward to some aesthetics >>> and high level picture of the process. Using Websockets, the dashboard can >>> be updated real time without the need of refreshing the page. >>> >>> Regards, >>> >>> Sivakumaran S >>> >>> >>> On 27-Aug-2016, at 10:10 AM, Mich Talebzadeh <mich.talebza...@gmail.com> >>> wrote: >>> >>> Thanks Sivakumaran >>> >>> I don't think we can use Zeppelin for this purpose. It is not a real >>> time dashboard or can it be. I use it but much like Tableau with added >>> Scala programming. >>> >>> Does anyone know of open source real time dashboards? >>> >>> >>> Cheers >>> >>> Dr Mich Talebzadeh >>> >>> >>> LinkedIn * >>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>> >>> >>> http://talebzadehmich.wordpress.com >>> >>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>> any loss, damage or destruction of data or any other property which may >>> arise from relying on this email's technical content is explicitly >>> disclaimed. The author will in no case be liable for any monetary damages >>> arising from such loss, damage or destruction. >>> >>> >>> >>> On 27 August 2016 at 09:42, Sivakumaran S <siva.kuma...@icloud.com> >>> wrote: >>> >>>> I would love to participate in developing a dashboard of some sort in >>>> lieu (or at least complement it) of Spark UI . >>>> >>>> Regards, >>>> >>>> Sivakumaran S >>>> >>>> On 27 Aug 2016 9:34 a.m., Mich Talebzadeh <mich.talebza...@gmail.com> >>>> wrote: >>>> >>>> Are we actually looking for a eal time dashboard of some sort for Spark >>>> UI interface? >>>> >>>> After all one can think a real time dashboard can do this! >>>> >>>> HTH >>>> >>>> >>>> >>>> Dr Mich Talebzadeh >>>> >>>> >>>> LinkedIn * >>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>>> >>>> >>>> http://talebzadehmich.wordpress.com >>>> >>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>>> any loss, damage or destruction of data or any other property which may >>>> arise from relying on this email's technical content is explicitly >>>> disclaimed. The author will in no case be liable for any monetary damages >>>> arising from such loss, damage or destruction. >>>> >>>> >>>> >>>> On 26 August 2016 at 23:38, Mich Talebzadeh <mich.talebza...@gmail.com> >>>> wrote: >>>> >>>> Thanks Jacek, >>>> >>>> I will have a look. I think it is long overdue. >>>> >>>> I mean we try to micro batch and stream everything below seconds but >>>> when it comes to help monitor basics we are still miles behind :( >>>> >>>> Cheers, >>>> >>>> Dr Mich Talebzadeh >>>> >>>> >>>> LinkedIn * >>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>>> >>>> >>>> http://talebzadehmich.wordpress.com >>>> >>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>>> any loss, damage or destruction of data or any other property which may >>>> arise from relying on this email's technical content is explicitly >>>> disclaimed. The author will in no case be liable for any monetary damages >>>> arising from such loss, damage or destruction. >>>> >>>> >>>> >>>> On 26 August 2016 at 23:21, Jacek Laskowski <ja...@japila.pl> wrote: >>>> >>>> Hi Mich, >>>> >>>> I don't think so. There is support for a UI page refresh but I haven't >>>> seen it in use. >>>> >>>> See StreamingPage [1] where it schedules refresh every 5 secs, i.e. >>>> Some(5000). In SparkUIUtils.headerSparkPage [2] there is >>>> refreshInterval but it's not used in any place in Spark. >>>> >>>> Time to fill an JIRA issue? >>>> >>>> What about REST API and httpie updating regularly [3]? Perhaps Metrics >>>> with ConsoleSink [4]? >>>> >>>> [1] https://github.com/apache/spark/blob/master/streaming/src/ma >>>> in/scala/org/apache/spark/streaming/ui/StreamingPage.scala#L158 >>>> [2] https://github.com/apache/spark/blob/master/core/src/main/sc >>>> ala/org/apache/spark/ui/UIUtils.scala#L202 >>>> [3] http://spark.apache.org/docs/latest/monitoring.html#rest-api >>>> [4] http://spark.apache.org/docs/latest/monitoring.html#metrics >>>> >>>> Pozdrawiam, >>>> Jacek Laskowski >>>> ---- >>>> https://medium.com/@jaceklaskowski/ >>>> Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark >>>> Follow me at https://twitter.com/jaceklaskowski >>>> >>>> >>>> On Thu, Aug 25, 2016 at 11:55 AM, Mich Talebzadeh >>>> <mich.talebza...@gmail.com> wrote: >>>> > Hi, >>>> > >>>> > This may be already there. >>>> > >>>> > A spark job opens up a UI on port specified by --conf >>>> "spark.ui.port=${SP}" >>>> > that defaults to 4040. >>>> > >>>> > However, on UI one needs to refresh the page to see the progress. >>>> > >>>> > Can this be polled so it is refreshed automatically >>>> > >>>> > Thanks >>>> > >>>> > >>>> > Dr Mich Talebzadeh >>>> > >>>> > >>>> > >>>> > LinkedIn >>>> > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJ >>>> d6zP6AcPCCdOABUrV8Pw >>>> > >>>> > >>>> > >>>> > http://talebzadehmich.wordpress.com >>>> > >>>> > >>>> > Disclaimer: Use it at your own risk. Any and all responsibility for >>>> any >>>> > loss, damage or destruction of data or any other property which may >>>> arise >>>> > from relying on this email's technical content is explicitly >>>> disclaimed. The >>>> > author will in no case be liable for any monetary damages arising >>>> from such >>>> > loss, damage or destruction. >>>> > >>>> > >>>> >>>> >>>> >>>> >>>> >>> >>> >> >
