Re: Get size of rdd in memory

2015-02-02 Thread Cheng Lian
qlContext.cacheTable(...) is strongly recommended for this use case.". If this is no longer accurate, i could make a PR to remove it. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Get-size-of-rdd-in-memory-tp10366p10392.html Sent from the Apa

Re: Get size of rdd in memory

2015-02-02 Thread ankits
http://apache-spark-developers-list.1001551.n3.nabble.com/Get-size-of-rdd-in-memory-tp10366p10392.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@

Re: Get size of rdd in memory

2015-02-02 Thread Cheng Lian
UI does not show us the size of the cached table. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Get-size-of-rdd-in-memory-tp10366p10388.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. ---

Re: Get size of rdd in memory

2015-02-02 Thread ankits
size of the cached table. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Get-size-of-rdd-in-memory-tp10366p10388.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

Re: Get size of rdd in memory

2015-01-30 Thread Cheng Lian
manager, or something like RDDInfo.fromRdd(rdd).memSize. I could use reflection or whatever to override the private access modifiers. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Get-size-of-rdd-in-memory-tp10366.html Sent from the Apache Spark Developers

Get size of rdd in memory

2015-01-30 Thread ankits
I understand i will need a reference to the block manager, or something like RDDInfo.fromRdd(rdd).memSize. I could use reflection or whatever to override the private access modifiers. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Get-size-of-rdd-in