[zfs-discuss] Occasional storm of xcalls on segkmem_zio_free

2012-06-06 Thread Sašo Kiselkov
So I have this dual 16-core Opteron Dell R715 with 128G of RAM attached to a SuperMicro disk enclosure with 45 2TB Toshiba SAS drives (via two LSI 9200 controllers and MPxIO) running OpenIndiana 151a4 and I'm occasionally seeing a storm of xcalls on one of the 32 VCPUs (>10 xcalls a second). Th

Re: [zfs-discuss] Occasional storm of xcalls on segkmem_zio_free

2012-06-06 Thread Richard Elling
On Jun 6, 2012, at 12:48 AM, Sašo Kiselkov wrote: > So I have this dual 16-core Opteron Dell R715 with 128G of RAM attached > to a SuperMicro disk enclosure with 45 2TB Toshiba SAS drives (via two > LSI 9200 controllers and MPxIO) running OpenIndiana 151a4 and I'm > occasionally seeing a storm of

Re: [zfs-discuss] Occasional storm of xcalls on segkmem_zio_free

2012-06-06 Thread Sašo Kiselkov
On 06/06/2012 04:55 PM, Richard Elling wrote: > On Jun 6, 2012, at 12:48 AM, Sašo Kiselkov wrote: > >> So I have this dual 16-core Opteron Dell R715 with 128G of RAM attached >> to a SuperMicro disk enclosure with 45 2TB Toshiba SAS drives (via two >> LSI 9200 controllers and MPxIO) running OpenIn

Re: [zfs-discuss] Occasional storm of xcalls on segkmem_zio_free

2012-06-06 Thread Sašo Kiselkov
On 06/06/2012 05:01 PM, Sašo Kiselkov wrote: > I'll try and load the machine with dd(1) to the max to see if access > patterns of my software have something to do with it. Tried and tested, any and all write I/O to the pool causes this xcall storm issue, writing more data to it only exacerbates it

Re: [zfs-discuss] Occasional storm of xcalls on segkmem_zio_free

2012-06-06 Thread Richard Elling
On Jun 6, 2012, at 8:01 AM, Sašo Kiselkov wrote: > On 06/06/2012 04:55 PM, Richard Elling wrote: >> On Jun 6, 2012, at 12:48 AM, Sašo Kiselkov wrote: >> >>> So I have this dual 16-core Opteron Dell R715 with 128G of RAM attached >>> to a SuperMicro disk enclosure with 45 2TB Toshiba SAS drives (v

Re: [zfs-discuss] Occasional storm of xcalls on segkmem_zio_free

2012-06-06 Thread Richard Elling
On Jun 6, 2012, at 8:22 AM, Sašo Kiselkov wrote: > On 06/06/2012 05:01 PM, Sašo Kiselkov wrote: >> I'll try and load the machine with dd(1) to the max to see if access >> patterns of my software have something to do with it. > > Tried and tested, any and all write I/O to the pool causes this xca

Re: [zfs-discuss] Occasional storm of xcalls on segkmem_zio_free

2012-06-06 Thread Jim Mauro
I can't help but be curious about something, which perhaps you verified but did not post. What the data here shows is; - CPU 31 is buried in the kernel (100% sys). - CPU 31 is handling a moderate-to-high rate of xcalls. What the data does not prove empirically is that the 100% sys time of CPU 31

Re: [zfs-discuss] Occasional storm of xcalls on segkmem_zio_free

2012-06-06 Thread Sašo Kiselkov
On 06/06/2012 09:43 PM, Jim Mauro wrote: > > I can't help but be curious about something, which perhaps you verified but > did not post. > > What the data here shows is; > - CPU 31 is buried in the kernel (100% sys). > - CPU 31 is handling a moderate-to-high rate of xcalls. > > What the data doe