Re: Use SparkListener to get overall progress of an action

2014-05-23 Thread Pierre B
>> Thanks >> >> Pierre >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256.html >> Sent from the Apache Spark User List mai

Re: Use SparkListener to get overall progress of an action

2014-05-23 Thread Philip Ogren
om/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Use SparkListener to get overall progress of an action

2014-05-23 Thread Pierre B
gt;>>> However, this was useful to gather some custom metrics (from within the >>>> tasks then) not really to collect overall monitoring information about the >>>> spark thingies themselves. >>>> For that Spark UI offers already a pretty good insight

Re: Use SparkListener to get overall progress of an action

2014-05-23 Thread Otávio Carvalho
e] >> >> <http://about.me/noootsab> >> >> >> On Thu, May 22, 2014 at 4:51 PM, Pierre B <> target="_top" rel="nofollow" link="external">[hidden email]> wrote: >> >>> Is there a simple w

Re: Use SparkListener to get overall progress of an action

2014-05-23 Thread Chester At Yahoo
rella >>>> about.me/noootsab >>>> >>>> >>>> >>>> >>>>> On Thu, May 22, 2014 at 4:51 PM, Pierre B <>>>> href="x-msg://7/user/SendEmail.jtp?type=node&node=6258&i=0" >>>>> targe

Re: Use SparkListener to get overall progress of an action

2014-05-23 Thread Pierre Borckmans
<>> href="x-msg://7/user/SendEmail.jtp?type=node&node=6258&i=0" >>> target="_top" rel="nofollow" link="external">[hidden email]> wrote: >>> Is there a simple way to monitor the overall progress of an action using

Re: Use SparkListener to get overall progress of an action

2014-05-23 Thread Mayur Rustagi
wrote: > >> Is there a simple way to monitor the overall progress of an action using >> SparkListener or anything else? >> >> I see that one can name an RDD... Could that be used to determine which >> action triggered a stage, ... ? >> >> >> Thanks &

Re: Use SparkListener to get overall progress of an action

2014-05-22 Thread Chester
g else? >> >> I see that one can name an RDD... Could that be used to determine which >> action triggered a stage, ... ? >> >> >> Thanks >> >> Pierre >> >> >> >> -- >> View this message in context: >> http://

Re: Use SparkListener to get overall progress of an action

2014-05-22 Thread andy petrella
email]> wrote: > >> Is there a simple way to monitor the overall progress of an action using >> SparkListener or anything else? >> >> I see that one can name an RDD... Could that be used to determine which >> action triggered a stage, ... ? >> >> &

Re: Use SparkListener to get overall progress of an action

2014-05-22 Thread Pierre B
me an RDD... Could that be used to determine which > action triggered a stage, ... ? > > > Thanks > > Pierre > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-acti

Re: Use SparkListener to get overall progress of an action

2014-05-22 Thread andy petrella
e, ... ? > > > Thanks > > Pierre > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Use SparkListener to get overall progress of an action

2014-05-22 Thread Pierre B
.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256.html Sent from the Apache Spark User List mailing list archive at Nabble.com.