Re: Can't see any thing one the storage panel of application UI

2014-08-05 Thread Andrew Or
ib, they don't persist the changed input , >> so I can't see the rdd from the web gui. >> >> I have changed the code and gained a 10x speed up. >> >> -- >> binbinbin915 >> Sent with Airmail >> >> >>

Re: Can't see any thing one the storage panel of application UI

2014-08-05 Thread Akhil Das
t; -- > binbinbin915 > Sent with Airmail > > > -------------- > View this message in context: Re: Can't see any thing one the storage > panel of application UI > <http://apache-spark-user-list.1001560.n3.nabble.com/Can-t-see-any-thing-one-the-storage-panel

Re: Can't see any thing one the storage panel of application UI

2014-08-04 Thread binbinbin915
Actually, if you don’t use method like persist or cache, it even not store the rdd to the disk. Every time you use this rdd, they just compute it from the original one. In logistic regression from mllib, they don't persist the changed input , so I can't see the rdd from the web gui. I have cha

Re: Can't see any thing one the storage panel of application UI

2014-08-04 Thread anthonyjschu...@gmail.com
Good idea Andrew... Using this feature allowed me to debug that my app wasn't caching properly-- the UI is working as designed for me in 1.0. It might be a good idea to say "no cached blocks" instead of an empty page... just a thought... On Mon, Aug 4, 2014 at 1:17 PM, Andrew Or-2 [via Apache Spa

Re: Can't see any thing one the storage panel of application UI

2014-08-04 Thread Andrew Or
Hi all, Could you check with `sc.getExecutorStorageStatus` to see if the blocks are in fact present? This returns a list of StorageStatus objects, and you can check whether each status' `blocks` is non-empty. If the blocks do exist, then this is likely a bug in the UI. There have been a couple of

Re: Can't see any thing one the storage panel of application UI

2014-08-04 Thread anthonyjschu...@gmail.com
I am (not) seeing this also... No items in the storage UI page. using 1.0 with HDFS... -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Can-t-see-any-thing-one-the-storage-panel-of-application-UI-tp10296p11361.html Sent from the Apache Spark User List mailing

Re: Can't see any thing one the storage panel of application UI

2014-07-21 Thread Preeti Khurana
Am getting the same issue . Spark version : 1.0 On 21/07/14 4:16 PM, "binbinbin915" wrote: >Hi, > >I'm running LogisticRegression of mllib. But I can't see the rdd >information >from storage panel. > > >