Spark executor memory information

2015-07-13 Thread Naveen Dabas
Hi, I am new to spark and need some guidance on below mentioned points: 1)I am using spark 1.2,is it possible to see how much memory is being allocated to an executor for web UI. If not how can we figure that out.2)    I am interested in source code of mlib,it is possible to get access to

Job aborted due to stage failure: Task not serializable:

2015-07-15 Thread Naveen Dabas
I am using the below code and using kryo serializer .when i run this code i got this error : Task not serializable in commented line2) how broadcast variables are treated in exceotu.are they local variables or can be used in any function defined as global variables. object StreamingLogIn