Re: CPU/Disk/network performance instrumentation

2014-07-09 Thread Surendranauth Hiraman
+1 on advanced tab. On Wed, Jul 9, 2014 at 5:20 PM, Mridul Muralidharan wrote: > +1 on advanced mode ! > > Regards. > Mridul > > On Thu, Jul 10, 2014 at 12:55 AM, Reynold Xin wrote: > > Maybe it's time to create an advanced mode in the ui. > > > > > > On Wed, Jul 9, 2014 at 12:23 PM, Kay Oust

Re: CPU/Disk/network performance instrumentation

2014-07-09 Thread Mridul Muralidharan
+1 on advanced mode ! Regards. Mridul On Thu, Jul 10, 2014 at 12:55 AM, Reynold Xin wrote: > Maybe it's time to create an advanced mode in the ui. > > > On Wed, Jul 9, 2014 at 12:23 PM, Kay Ousterhout > wrote: > >> Hi all, >> >> I've been doing a bunch of performance measurement of Spark and, a

Re: CPU/Disk/network performance instrumentation

2014-07-09 Thread Shivaram Venkataraman
I think it would be very useful to have this. We could put the ui display either behind a flag or a url parameter Shivaram On Wed, Jul 9, 2014 at 12:25 PM, Reynold Xin wrote: > Maybe it's time to create an advanced mode in the ui. > > > On Wed, Jul 9, 2014 at 12:23 PM, Kay Ousterhout > wrote:

Re: CPU/Disk/network performance instrumentation

2014-07-09 Thread Reynold Xin
Maybe it's time to create an advanced mode in the ui. On Wed, Jul 9, 2014 at 12:23 PM, Kay Ousterhout wrote: > Hi all, > > I've been doing a bunch of performance measurement of Spark and, as part of > doing this, added metrics that record the average CPU utilization, disk > throughput and utili

CPU/Disk/network performance instrumentation

2014-07-09 Thread Kay Ousterhout
Hi all, I've been doing a bunch of performance measurement of Spark and, as part of doing this, added metrics that record the average CPU utilization, disk throughput and utilization for each block device, and network throughput while each task is running. These metrics are collected by reading t