Hey Roman,
Ya definitely checkout pull request 42 - one cool thing is this patch
now includes information about in-memory storage in the listener
interface, so you can see directly which blocks are cached/on-disk
etc.
- Patrick
On Mon, Mar 17, 2014 at 5:34 PM, Matei Zaharia wrote:
> Take a look
Take a look at the SparkListener API included in Spark, you can use it to
capture various events. There’s also this pull request:
https://github.com/apache/spark/pull/42 that will persist application logs and
let you rebuild the web UI after the app runs. It uses the same API to log
events.
Ma
Hi.
We're thinking about writing a tool that would read Spark logs and output
cache contents at some point in time (e.g. if you want to see what data
fills the cache and whether some of it may be unpersisted to improve
performance).
Are there similar projects that already exist? Is there a list o
