Adam Cheal wrote:
Hi Adam,
thanks for this info. I've talked with my colleagues
in Beijing (since
I'm in Beijing this week) and we'd like you to try
disabling MSI/MSI-X
for your mpt instances. In /etc/system, add
set mpt:mpt_enable_msi = 0
then regen your boot archive and reboot.
I had already done this at Mark Johnson's request, though I had just added:
set mpt:mpt_enable_msi=0
set mptsas:mptsas_enable_msi=0
...to the /etc/system file and did a full reboot. I didn't know I had to
regen the boot archive manually for those new settings to take effect:
how/why would I do this? The fact the IO rate changed for me during the test
indicated that the new settings had "taken". Long story short, I still had
the problems after making this change though they took longer to appear.
I thought you had just set
set xpv_psm:xen_support_msi = -1
which is different, because that sets the xen_support_msi variable
which lives inside the xpv_psm module.
Setting mptsas:* will have no effect on your system if you do not
have an mptsas card installed. The mptsas cards are not generally
available yet (they're 2nd generation), so I would be surprised if
you had one.
Longer story:
After making the change, rebooting and starting a scrub on the pool I
watched iostat for hints of trouble (i.e. error column changes). The IO
rate to the disk was definately slower after this change, with individual
disks never getting more than 50% busy and 2 active commands. About three
hours later, the read errors/bus resets started to appear. I assume the
longer delay before errors was just because the reduced IO was putting
less of a strain on the driver/hardware. Let me know if you want me to
refine this test or any other diagnostics that would help you out.
I think that's sufficient to go on for the moment, thankyou.
cheers,
James C. McPherson
--
Senior Kernel Software Engineer, Solaris
Sun Microsystems
http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
