Re: Measure the Total Network I/O, Cpu and Memory Consumed by Spark Job

2014-04-14 Thread yxzhao
il]<http://user/SendEmail.jtp?type=node&node=4214&i=0> > > wrote: > >> Hi All, >> >> I am intrested in measure the total network I/O, cpu and memory >> consumed by Spark job. I tried to find the related information in logs and >> Web UI. Bu

Re: Measure the Total Network I/O, Cpu and Memory Consumed by Spark Job

2014-04-14 Thread dachuan
I haven't tried before, but it seems Ganglia could be the tool. On Wed, Apr 2, 2014 at 6:40 PM, yxzhao wrote: > Hi All, > > I am intrested in measure the total network I/O, cpu and memory > consumed by Spark job. I tried to find the related information in logs and &g

Measure the Total Network I/O, Cpu and Memory Consumed by Spark Job

2014-04-02 Thread yxzhao
Hi All, I am intrested in measure the total network I/O, cpu and memory consumed by Spark job. I tried to find the related information in logs and Web UI. But there seems no sufficient information. Could anyone give me any suggestion? Thanks very much in advance. -- View this