Re: [OpenIndiana-discuss] cryptic device naming?
The other variable here is the controller settings. The LSI card can use persistent mappings forthe disks, so it's OS presented device is constant, regardless of what slot it is physically in. This drove me nuts trying to figure out what was going on the first time I encountered it. I now use lsiutil to make sure it is off. Most cards seem to now have it off by default. The Disk Bay Indicator function is still elusive. On my Supermicro hardware I have both the ses and ipmi drivers from the last sxce release working with oi147, and a borrowed sestopo to confirm it is all visible. Unfortunately it still didn't help the fm drivers turn on the leds yet. I know it all works, as I have had the full "Sun Storage Software" running on it, and the locate works perfectly. On 06/11/2010 12:36 p.m., McBofh wrote: On 6/11/10 07:41 AM, Roy Sigurd Karlsbakk wrote: You've _got_ cXtYdZ naming - it's just that the (Y) part is based on the target-port name property, which is generated from the device devid (closely related to the GUID). With your hba, which is using mpt_sas, you have MPxIO on by default, which is the direction that ON has been heading in for many years. Is it possible to turn mpxio off without ruining the rpool? Yes, but I don't recommend it. There's been a concerted effort to make MPxIO the default operating mode for several years, and with mpt_sas that was one of the driver design criteria. Also, may this help me get around the problem with identifying the drives? Possibly - depends on what the mpt_sas hba does under the hood. I don't recall whether it does the same thing as the mpt controller, then your chance of getting a "logical target-id" rather than a physical, hard, slot/bay number is slim. What you could do, otoh, is probe your ses device, look at the output from SES diagnostic pagecode 0xa, Additional Element Status and hope that the Element Index Present (EIP) bit is set to 1 in the pagecode response. If it is, then you can match up bay numbers with element index entries. Or you could run /usr/lib/fm/fmd/fmtopo -dV and see if you can get output like this: hc://:product-id=SUN-Storage-J4200:server-id=:chassis-id=0848QAJ001:serial=0820T4LXSA3LM4LXSA:part=SEAGATE-ST330055SSUN300G:revision=0B92/ses-enclosure=1/bay=0/disk=0 group: protocol version: 1 stability: Private/Private resource fmri hc://:product-id=SUN-Storage-J4200:server-id=:chassis-id=0848QAJ001:serial=0820T4LXSA3LM4LXSA:part=SEAGATE-ST330055SSUN300G:revision=0B92/ses-enclosure=1/bay=0/disk=0 label string SCSI Device 0 FRU fmri hc://:product-id=SUN-Storage-J4200:server-id=:chassis-id=0848QAJ001:serial=0820T4LXSA3LM4LXSA:part=SEAGATE-ST330055SSUN300G:revision=0B92/ses-enclosure=1/bay=0/disk=0 ASRU fmri dev:///:devid=id1,s...@n5000c5000b20566b//scsi_vhci/d...@g5000c5000b20566b group: authority version: 1 stability: Private/Private product-id string SUN-Storage-J4200 chassis-id string 0848QAJ001 server-id string group: storage version: 1 stability: Private/Private logical-disk string c0t5000C5000B20566Bd0 manufacturer string SEAGATE model string ST330055SSUN300G serial-number string 0820T4LXSA 3LM4LXSA firmware-revision string 0B92 capacity-in-bytes string 3000 target-port-l0ids string[] [ "w5000c5000b205669" ] group: io version: 1 stability: Private/Private devfs-path string /scsi_vhci/d...@g5000c5000b20566b devid string id1,s...@n5000c5000b20566b phys-path string[] [ "/p...@0,0/pci8086,3...@2/pci8086,3...@0/pci8086,3...@1/pci1000,3...@0/d...@19,0" ] You should be able to see useful information by running # cfgadm -lav eg, Ap_Id Receptacle Occupant Condition Information When Type Busy Phys_Id c3 connected configured unknown unavailable scsi-sas n /devices/p...@0,0/pci10de,3...@a/pci1000,3...@0:scsi c3::0,0 connected configured unknown Client Device: /dev/dsk/c5t5000CCA00510A7CCd0s0(sd37) I tried that, and I got some of the same results. However, the sdXX doesn't mape to the device port, but seem to map to some (to me) random drive. I also recommend reviewing my presentation on devids and GUIDs: http://www.jmcp.homeunix.com/~jmcp/WhatIsAGuid.pdf ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] cryptic device naming?
LSI web site Same tool for all FC and SAS adapters. Mark. On 16/11/2010 11:49 p.m., Roy Sigurd Karlsbakk wrote: - Original Message - The other variable here is the controller settings. The LSI card can use persistent mappings forthe disks, so it's OS presented device is constant, regardless of what slot it is physically in. This drove me nuts trying to figure out what was going on the first time I encountered it. I now use lsiutil to make sure it is off. Most cards seem to now have it off by default. Any idea where I can find this tool? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] cryptic device naming?
lsiutil version 1.63 should work with the SAS2 series, but is a bit harder to find. I have a copy at work and will PM for you to try. Mark. On 19/11/2010 8:47 a.m., Roy Sigurd Karlsbakk wrote: Tried that - told me "no card found". Seems it doesn't support the LSI SAS2 card - Original Message - LSI web site Same tool for all FC and SAS adapters. Mark. On 16/11/2010 11:49 p.m., Roy Sigurd Karlsbakk wrote: - Original Message - The other variable here is the controller settings. The LSI card can use persistent mappings forthe disks, so it's OS presented device is constant, regardless of what slot it is physically in. This drove me nuts trying to figure out what was going on the first time I encountered it. I now use lsiutil to make sure it is off. Most cards seem to now have it off by default. Any idea where I can find this tool? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ZFS improvements.
With WD EARS disks, since they do not expose the 4K size to the OS, there is not much to be gained with their internal 4k sector size. Performance and reliability with these is also poor. The fixed, large TLER defaults cause constant dropouts, and I have found from experience that these disks don't work well with ZFS. Mark. On 24/11/2010 7:08 a.m., Jesus Cea wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23/11/10 18:55, Maurilio Longo wrote: mauri...@rsync:~$ ls -lR /nas/test/snv_143/ | wc -l 175184 At 2Kb, mean, per file, we have 360Mb, more or less, but here we go from 2,73Gb to 3,81 so it is more than 1Gb, or, nearly 8Kb per file! If the znodes are 512 bytes too, they waste 3584 in a 4KB harddisk. So adding both you have extra 5632 bytes wasted per file. Do you use the same blocksize?. Same compression?. Anyway, I think that ZFS will try to align everything to 4KB, so it is going to waste space for every on-disk datastructure. - -- Jesus Cea Avion _/_/ _/_/_/_/_/_/ j...@jcea.es - http://www.jcea.es/ _/_/_/_/ _/_/_/_/ _/_/ jabber / xmpp:j...@jabber.org _/_/_/_/ _/_/_/_/_/ . _/_/ _/_/_/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/_/_/ _/_/_/_/ _/_/ "My name is Dump, Core Dump" _/_/_/_/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTOwDBZlgi5GaxT1NAQJP5gQAi6FG5kJXYCQ2EdeRD7s8ZOcwqzH6c/zY k84N8yuo4iPom7mFVDYKPO/mgIAS5+OqRcP0DLmbkxYyOvEWuCG7Ih7LbYWhjQxB PTWIBtYfyYehwwiH+izaSCjKOu1L9TKfUxhLCnLnevmfKc9qdH5SZ1tH7qTCnAYY 2IvP427j6KA= =pe9v -END PGP SIGNATURE- ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] Text Installer creates rpool1
I tried the OI Build147 text installer for the first time, and ended up with rpool1 instead of the expected rpool. Is this normal ? The disks had an existing pool, but I selected use whole disk and overwrite. I have done this many times with OpenSolaris versions and never encountered it before. Overall the text install didn't help my problem, since it offeres no static network setup. The other headache was getting static ip's setup. The "old way" with hostname.intf etc. produced a interface to nowhere. This is what triggered the reinstall. The ipadm method did work, once I figured out what was needed. The storage box has 6xGb and 2x10Gb nics, so plenty to configure. One difference I found from dev-134 is it doesn't play well with an ip on the same subnet on two different interfaces. If one is down, the other can't pass packets either. Other than that, it has been stable and performing ok during testing with Intel SSD's for logs and 32x Seagate 2Tb SAS drives. I have Fault Management working nicely with net-snmp, and even an old bmc driver from the last SE released talking to the IPMI card. It didn't seem to work through to FM though. I need to get a build/development environment before I can follow that up. I will also have a poke around in SE2011 and see if it has any improvements in this area I can borrow. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Text Installer creates rpool1
On 02/12/2010 2:11 a.m., Roy Sigurd Karlsbakk wrote: I tried the OI Build147 text installer for the first time, and ended up with rpool1 instead of the expected rpool. As mentioned by Jeppe, destroying the old rpool will be a workaround for this. I guess what happens is that OI boots up, sees the rpool name in use and chooses rpool1. The other headache was getting static ip's setup. The "old way" with hostname.intf etc. produced a interface to nowhere. Disable nwam and switch back to the old way: # svcadm disable svc:/network/physical:nwam # svcadm enable svc:/network/physical:default The configure hostname.nicname as before Hi Roy, That is what my post-install script does, and has worked well until now. This is my first storage server build with the OpenIndiana release. The nic was pingable, but the network was not. dladm showed link, and packet tx and rx counts changed, but no joy connecting to the lan. I've never encountered this before after many installs over the last few years. After a reinstall, I used ipadm instead, as it is much easier, and everything was ok. Other than that, it has been stable and performing ok during testing with Intel SSD's for logs and 32x Seagate 2Tb SAS drives. Out of curiosity, what controllers are you using? And what device naming are you getting? The old c0t0d0 or the newer c0tWWNd0? I get the latter on LSI 9211-8i controllers, and there, the device order fails to mirror the chassis order, with supermicro chassis (and even with direct attach). Supermicro AOC-USAS-L8i (LSI 1068) with IT mode firmware 1.30. Old style naming convention, since it is a single port backplane. I think the name is more about the multipath driver being present, which needs to use wwn for dual paths. The only issue on device order is I don't get the first two slots as the first two drives on a cold boot, but do from a soft boot or reset. The first IT mode controller I worked with had persistence enabled after I changed the firmware to IT. This may be different for the SAS2. I'll have the Supermicro equivalent (AOC-USAS2-L8e) to try in about a month. It may be worth asking LSI about the issue. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] [oi-dev] Proposal: OpenIndiana Stable Branch
+1 to a minimal server version. (late reply - two weeks of computer free holiday) For web based management, it would make more sense to work on Webmin modules to add the missing bits like zfs. A postfix default mailer would also get my vote. As my focus is on nfs/cifs storage systems, I currently use the standard text installer, and a custom post-install script to bend it into shape. I have previously used a fully custom install script and build everything from scratch via network. Currently I'm determined to achieve read-only USB/DOM boot with raidz root. Having already ported the fishworks storage os to generic hardware for fun, it should be too hard. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Proposal: OpenIndiana Stable Branch
Postfix is probably the easiest drop-in replacement. But IMO a packaging of it should get lots of testing before going into a stable distro, and regardless of which is eventually the default or preferred choice, both should remain available. I did poke around at this, but found that the fmd smtp notification uses sendmail, and has a dependancy on it, so I put the effort into getting fmd working via snmp instead. I'm probably biased, having had to hire a "sendmail expert" for a week to create a complex email routing server with Solaris, that I later replaced with postfix myself in an afternoon (on Centos). I'm a fan of the minimal "fries with that" OS approach, and then clip in your favourite packages. I'm about to "update" a 40Tb snv_134 storage server to OpenIndiana. I've migrated the data already, and there is a considerable difference in setup around networking and zfs ACL's especially with sharing filesystems with both nfs and smb. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Build 147 ZFS / NFS / iSCSI system reboot
Hi Steve, some more info on the shelves would help. SCSI, FC , SAS, brands etc. Mark. On 17/02/2011 3:19 p.m., Steve Jacobson wrote: All, We have some systems that we're prototyping that use an OpenIndiana (b147) server with two disk shelves running Ubuntu linux 10.10 and iSCSI. When we work with a single shelf with the head system, everything works perfectly. When we add a second shelf, we are seeing frequent server reboots. There is nothing showing up in /var/crash when this happens. The last message that shows up in /var/adm/messages is: Feb 15 15:58:08 inp-production-zfs-archive-10 scsi: [ID 107833 kern.warning] WARNING: /iscsi/d...@iqn.2007-07.com.doyenz%3Ainp-production-dshelf-09.megaraid1,4 (sd19): Feb 15 15:58:08 inp-production-zfs-archive-10 incomplete write- retrying Feb 15 16:04:22 inp-production-zfs-archive-10 genunix: [ID 429879 kern.notice] ^MOpenIndiana Build oi_147 64-bit There are no disk or other types of errors showing up in the Linux systems' logs. When the head system comes back, it appears to be just fine, but at some point in the future, when there is i/o, it will reboot again. Has anyone seen behavior like this before, or does anyone have any advice on how to debug / resolve this? Thanks! -Steve J -- Steve Jacobson | Director of Operations | Doyenz, Inc. 11245 SE 6th Street, Suite 120 | Bellevue, Wash. | 98004 Main: 206.905.4713 | Mobile: 206.310.7760 | Fax: 206.260.9129 ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Build 147 ZFS / NFS / iSCSI system reboot
On 18/02/2011 10:19 a.m., Steve Jacobson wrote: Sure - the shelves are SuperMicro systems running Ubuntu 10.10. There are 36 Seagate Constellation ES 2TB drives connected to 5 LSI ELP controllers. The drives are arranged in four groups of eight drives in RAID6 and one group of four drives in RAID5. These five RAID groups are set up as iSCSI targets on the Ubuntu system. There are two of these SuperMicro shelves. So, there are ten LUNs available by iSCSI to OpenIndiana. The tank is created with one VDEV for the ten LUNs. Sas or Sata version of drive, which model chassis and dual or single port connections? ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] It just trashed itself!!
I had an interesting issue today with one of my Open Indiana storage servers. It has around 15 smb/nfs shares and 40Tb of storage. The problems may have slowly crept up on it, as logs from the nfs client showed slow response issues starting about 12 hours earlier. Eventually it had ground to a halt, and would not complete a console login. I achieved a normal shut-down via the power button, but on reboot it was somewhat stuffed. On power up, it dropped into single user mode, due to networking issues. A 'dladm show-phys' revealed some missing network devices. The box has two to on-board and a quad gigabit card as igb devices, as well as a dual 10Gbit ixgbe, but only 3 x igb and 1 x ixgbe devices showed up. I tried another reboot, but that didn't help much either, as some were still missing. Then a reboot - -r, and that resulted in all the network devices disappearing. Suspecting possible hardware issues, I booted of the text installation cdrom, and found all the network devices were present and correct. A zpool import & scrub of the OS mirror showed no issues either. About an hour later, after a full OS reinstall and reconfigure, it was back up in production, thanks to the real virtues of zfs - recovery and portability, with smb and nfs shares intact. (I have build a raw vm workstation Open Solaris on a sata disk , moved it to an AMD and then Intel processor box, and had no problems just booting it up) I've saved one of the mirrored OS disks for a post-mortem, to try to find out what happened. Some of the errors on screen suggested write issues to some /dev/ devices, but when a production system is down, rapid recovery is always the primary goal, and analysis took a back seat. I've been slowly, (try moving 40Tb in a hurry and keeping data available), upgrading the Open Solaris boxes to Open Indiana to resolve the scrub impact and some of the other issues I had encountered. These have been very reliable for up to two years so far. The oldest has been up for about a year, but this one only a month. Hopefully this isn't a regular event, but I may keep a pre-built OS disk ready just in case. If anyone has suggestions on what to look for in the wreckage, it would be helpful. Mark. [Sparing a thought for Christchurch Earthquake victims. Thankfully, my family there are all safe.] ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] It just trashed itself!!
On 26/02/2011 7:09 a.m., David Brodbeck wrote: On Fri, Feb 25, 2011 at 5:53 AM, Lou Piccianowrote: Mark, It may not help at all - but what kind of network interface hardware are you using? igb is the Broadcom driver. No, igb is Intel. In this case the 82576EB chipset for the gigabit. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Solaris Express server name broadcast
Ok, so lets get back on to the subject. As far as I am aware, the solaris kernel based cifs doesn't appear to support the same "server" features of Samba, but acts more like a dumb "workstation". This will mean it takes some time for something else on the network to assume the master browser role, and maintain a browse list. Much like a bunch of Windows 95's in a workgroup. The use of static hosts entries may help resolve names more quickly, but not populate brower lists. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] How old is openindiana ?
My installation must be "very" ancient after this resilver. pool: rpool state: ONLINE scan: resilvered 18.7G in 307445734561825847h23m with 0 errors on Mon Feb 7 12:28:47 2011 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0ONLINE 0 0 0 c1t8d0s0 ONLINE 0 0 0 c1t9d0s0 ONLINE 0 0 0 Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] [illumos-Developer] SES support for Super Micro chassises?
On 10/03/2011 9:33 p.m., Cyril Plisko wrote: On Thu, Mar 10, 2011 at 1:15 AM, Jason King wrote: You'd probably need to write a C program that opens the device (assuming one is created on the system) and use the uscsi interface to issue the commands as ioctls. Or use sg_senddiag from sg3_utils package. With the LSI IT mode controllers and Supermicro SAS Expander backplane (E1,E2,E16,E26) you should see /dev/es/ses0 etc. OpenIndianan SunOS pod03-sss 5.11 oi_147 i86pc i386 i86pc Solaris c1::es/ses0ESI connectedconfigured unknown c1::es/ses1ESI connectedconfigured unknown c1::smp/expd0 smp connectedconfigured unknown c1::smp/expd1 smp connectedconfigured unknown You can use sestopo pkg install system/io/tests sestopo /dev/es/ses0 This is some of what you get, but even whit this, Fault Management doesn't do anything interesting with it. This server also has an IPMI card, so ipmitopo also lists this info (see end of post) Mark. Node Type: 1 nvlist version: 0 scsi-inquiry-vendor = LSILOGIC scsi-inquiry-product = SASX28 A.0 scsi-inquiry-revision =9 Node Type: 2 nvlist version: 0 ses-enclosure-id = 0x0 ses-enclosure-service-proc-id = 0x0 ses-enclosure-service-proc-count = 0x0 ses-logical-id = (embedded nvlist) nvlist version: 0 naa-id-integer = 0x50030480003bbf7f naa-id-type = 0x5 naa-company-id = 0x3048 naa-vendor-specific-a = 0x3bbf7f (end ses-logical-id) ses-vendor-id = LSILOGIC ses-product-id = SASX28 A.0 ses-product-revision = 9 ses-element-index = 0x12 ses-element-only-index = 0x10 ses-element-type = 0xe ses-class-description = Enclosure ses-status-code = 0x1 ses-swapped = 0 ses-disabled = 0 ses-failure-predicted = 0 ses-identify = 0 ses-warning = 1 ses-failed = 0 ses-power-cycle-delay = 0x0 ses-warning-indicator-requested = 0 ses-failure-indicator-requested = 0 ses-power-cycle-duration = 0x0 ses-description = BOX 001 ses-microcode-dl-status = 0x0 ses-microcode-dl-addl-status = 0x0 ses-microcode-maximum-size = 0x4 ses-microcode-buffer-id = 0xff ses-microcode-buffer-offset = 0x0 libses-element-type-name = ENCLOSURE libses-chassis-serial = 50030480003bbf7f libses-internal = 1 Node Type: 4 Element Type: ARRAY_DEVICE nvlist version: 0 ses-element-index = 0x0 ses-element-type = 0x17 ses-class-description = Array Device ses-status-code = 0x0 ses-swapped = 0 ses-disabled = 0 ses-failure-predicted = 0 ses-remap-rebuild-abort = 0 ses-remap-rebuild = 0 ses-in-failed-array = 0 ses-in-critical-array = 0 ses-consistency-check = 0 ses-hot-spare = 0 ses-reserved-device = 0 ses-ok = 0 ses-reported-via = 0 ses-identify = 0 ses-ready-to-remove = 0 ses-ready-to-insert = 0 ses-enclosure-bypassed-b = 0 ses-enclosure-bypassed-a = 0 ses-do-not-remove = 0 ses-app-client-bypassed-a = 0 ses-device-bypassed-b = 0 ses-device-bypassed-a = 0 ses-bypassed-b = 0 ses-bypassed-a = 0 ses-off = 0 ses-fault-requested = 0 ses-fault-sensed = 0 ses-app-client-bypassed-b = 0 ses-description = Array Device libses-element-type-name = ARRAY_DEVICE Node Type: 8 Element Type: ARRAY_DEVICE nvlist version: 0 ses-element-index = 0x1 ses-element-only-index = 0x0 ses-element-class-index = 0x0 ses-element-type = 0x17 ses-status-code = 0x1 ses-swapped = 0 ses-disabled = 0 ses-failure-predicted = 0 ses-remap-rebuild-abort = 0 ses-remap-rebuild = 0 ses-in-failed-array = 0 ses-in-critical-array = 0 ses-consistency-check = 0 ses-hot-spare = 0 ses-reserved-device = 0 ses-ok = 0 ses-reported-via = 0 ses-identify = 0 ses-ready-to-remove = 0 ses-ready-to-insert = 0 ses-enclosure-bypassed-b = 0 ses-enclosure-bypassed-a = 0 ses-do-not-remove = 0 ses-app-client-bypassed-a = 0 ses-device-bypassed-b = 0 ses-device-bypassed-a = 0 ses-bypassed-b = 0 ses-bypassed-a = 0 ses-off = 0 ses-fault-requested = 0 ses-fault-sensed = 0 ses-app-client-bypassed-b = 0 ses-description = 000 ses-sas-not-all-phys = 1 ses-bay-number = 0x0 ses-sas-phys = (array of embedded nvlists) (start ses-sas-phys[0]) nvlist
Re: [OpenIndiana-discuss] NFS hang during copy
Hi Roy, I have up to 30 nfs v3 mounts from each of four Centos 5.5 boxes to two OI148 targets and haven't seen that sort of issue. Mark. On 21/03/2011 7:04 a.m., Chris Ridd wrote: On 20 Mar 2011, at 14:49, Roy Sigurd Karlsbakk wrote: Hi all I'm fighting a problem with an OpenIndiana 148 server and NFS3 mounts from Linux clients. A simple cron job is run that moves some data files from another server to the OI box. This runs well for a while, until at some point, the client hangs and reports NFS server connection failure. The calltrace from linux is [...] What mount options is the Linux client using? Chris ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] OpenIndiana hosting advocating?
On 25/03/2011 12:53 p.m., Christopher Chan wrote: On Friday, March 25, 2011 01:06 AM, Jesus Cea wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 When I check around for hosting services (I do it a lot, part of my job) I always check availability of Solaris like OSs. I must say that the result is sad. Not withstanding available hosters, it should be kinda hard to find a hoster when the/some devs are calling chaps who use OI in production idiots and to wait for a stable release? Since that take casts me amongst the "idiots", I beleive it has always been the case that a company adopts what it can best support. One end of that is windows, thru Linux to Solaris at the other extreme. Personally I'd rate the other end in the "idiot" category, but then that is only my [expert] opinion and YMMV. When I find another way that can let me recover Tb of nfs and cifs file systems from an OS rebuild, in under 60 minutes, then my opinion might change. Until then, it is still the best ZFS platform I have available. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] New illumos based OpenIndiana test repo
On 29/03/2011 4:06 a.m., Alasdair Lumsden wrote: Hi All, This is an internal test and development repo for a forthcoming release; it's not a release in and of itself, so please don't update your day to day workstations to this. Only install this if you're interested in developing OpenIndiana+Illumos and know what you're doing. If you don't know what this repo is, please don't use it. (We should probably have announced this to oi-dev rather than oi-discuss - sorry!) Please don't apologise for keeping the faithful informed. Cheers, Alasdair ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] reboot/shutdown commands (Was: Re: OI boot problem)
On 29/03/2011 4:44 a.m., Volker A. Brandt wrote: Jonathan Adams writes: If you're using a server then expect to know some of the commands for running the server. If you're using the desktop there is a nice graphical "Shutdown" routine. If you're in front of the hardware and you want to power down (and it's an ATX case) press the power button once. Couldn't have said it better. +99 :-) And when you didn't really mean to press that power button, press it twice and it will cancel, if it can . ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] pool messup
Hi Roy, The drive order issue is likely to be persistent mapping. It can be disabled using lsiutil. I found it was enabled by default on early cards, and disabled on later ones, but changing the firmware doesn't change the origional setting. What model are the disks ? Did the origional system also use expanders ? Mark. On 12/04/2011 7:33 a.m., Roy Sigurd Karlsbakk wrote: This is an entry level proliant, very standard gear. I cannot believe that you would be the first one using them with ZFS (those sell really well) Is the Jbod configured? I mean, are all the the drives correctly defined in whatever config utility provided at boot time? Yes Are the drives in both chassis failing? Yes Supermicro is affordable and pretty basic but I fear using them in any other way than what they were designed for. Is there something on those chassis that you were supposed to configure? Those devices are designed and tested with very traditional raid an jbod setups in mind... think if there is something you didnt do but you would need to do in case of setting up a traditional jbod storage... We use supermicros on some rather largish servers, and they're very stable. Thee issue here is probably a driver or controller issue, see http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg32079.html Back when I had my problem I was testing all the drives 1 by 1 and all the cables, and all the different combinations of things... it was all useless. when something so fishy appears beware of the controllers. To make a memtest is a good idea, you could let it running over night, if memory corrupts probably there is some hardware problem in the motherboard. Thats all that comes to my mind... is the hardrive order swapping between reboots? But my bet is supermicro, or the way you configured the jbod utility. The controller is running a firmware without RAID support, so everything's JBOD. The storage chassises don't have any configuration, they just have a SAS expander, that's all. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] pool messup
On 12/04/2011 8:38 a.m., Roy Sigurd Karlsbakk wrote: Persistent mapping is disabled, and has been for some time. lsiutil shows no persistent mappings. The drives are a varity of makes. The system has been the same since I reinstalled it a month or two back. roy I would suggest using lsiutil to check the port link speed of the whole SAS topology. You can see the link status and negotiated speed for each port on the expander. A poor SAS cable connection to a JBOD can affect connectivity and reliability, as can incorrect negotiation of speed. Moving a controller to a different slot will change the device id's. I have experienced disk issues when under high load. In my experience, very few disks work well with expanders. I use only one model after having issues show up after months of trouble free running. For Supermicro expanders, avoid all 6Gbit sata disks and all non RE WD. A few Seagates are ok e.g. ST32000542AS , but ST32000641NS (6gbit) are not. Hitachi disks seem to be a winner all round, but now that WD is buying them out, it will likely that they will soon disappear. Bad or suspect blocks can also show up as a problem with timeouts. The zfs timeout issue is usually the result of the long error retry that nearly all desktop (except Hitachi) disks use by default. My take on this is the manufacturers method of forcing you to use Enterprise disks, since this behaviour breaks most raid systems. The end user used to have control of the timeout, but it has now been hard coded to very long. - Original Message - Hi Roy, The drive order issue is likely to be persistent mapping. It can be disabled using lsiutil. I found it was enabled by default on early cards, and disabled on later ones, but changing the firmware doesn't change the origional setting. What model are the disks ? Did the origional system also use expanders ? Mark. On 12/04/2011 7:33 a.m., Roy Sigurd Karlsbakk wrote: This is an entry level proliant, very standard gear. I cannot believe that you would be the first one using them with ZFS (those sell really well) Is the Jbod configured? I mean, are all the the drives correctly defined in whatever config utility provided at boot time? Yes Are the drives in both chassis failing? Yes Supermicro is affordable and pretty basic but I fear using them in any other way than what they were designed for. Is there something on those chassis that you were supposed to configure? Those devices are designed and tested with very traditional raid an jbod setups in mind... think if there is something you didnt do but you would need to do in case of setting up a traditional jbod storage... We use supermicros on some rather largish servers, and they're very stable. Thee issue here is probably a driver or controller issue, see http://www.mail-archive.com/zfs-discuss@opensolaris.org/msg32079.html Back when I had my problem I was testing all the drives 1 by 1 and all the cables, and all the different combinations of things... it was all useless. when something so fishy appears beware of the controllers. To make a memtest is a good idea, you could let it running over night, if memory corrupts probably there is some hardware problem in the motherboard. Thats all that comes to my mind... is the hardrive order swapping between reboots? But my bet is supermicro, or the way you configured the jbod utility. The controller is running a firmware without RAID support, so everything's JBOD. The storage chassises don't have any configuration, they just have a SAS expander, that's all. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Device placement havoc (Was: pool messup)
On 15/04/2011 6:33 a.m., Roy Sigurd Karlsbakk wrote: So, back to square one, how the hell can I find a mapping between disk and slot? Some say this is supposed to work on Nexenta, with the same hardware, but has Nexenta released the code for this? Do they have to release the code, as would be the case with GPL? Will they release any code if asked? Here's more info - output from prtpicl and fmtopo: http://pastebin.com/xtJ7R0TY Thanks Roy, have you tried sestopo (/usr/lib/scsi/sestopo) ? gs_ses from sg3_utils also works. install pkg system/io/tests for sestopo. ipmitopo, with an ipmi card installed and functional ses, will also list bay and device info. This is from a Supermicro 3U E1 chassis. hc://:product-id=X7DW3:server-id=pod02-sss:chassis-id=0123456789:serial=BLA0P87003VP:part=FUJITSU-MBA3147RC:revision=0103/chassis=0/bay= 0/disk=0 group: protocol version: 1 stability: Private/Private resource fmri hc://:product-id=X7DW3:server-id=pod02-sss:chassis-id=0123456789:serial=BLA0P87003VP:part=FUJITSU-MBA314 7RC:revision=0103/chassis=0/bay=0/disk=0 label string000 FRU fmri hc://:product-id=X7DW3:server-id=pod02-sss:chassis-id=0123456789:serial=BLA0P87003VP:part=FUJITSU-MBA314 7RC:revision=0103/chassis=0/bay=0/disk=0 ASRU fmri dev:///:devid=id1,sd@n50e01c67a640//pci@0,0/pci8086,4023@3/pci15d9,a680@0/sd@8,0 group: authority version: 1 stability: Private/Private product-idstringX7DW3 chassis-idstring0123456789 server-id stringpod02-sss group: storageversion: 1 stability: Private/Private logical-disk stringc1t8d0 manufacturer stringFUJITSU model stringMBA3147RC Mark. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] [zfs-discuss] aclmode -> no zfs in heterogeneous networks anymore?
On 26/04/2011 10:39 p.m., Gregory Youngblood wrote: I haven't been bit by this yet, but think I am about to be as I was planningvon setting up a share that would be used across Mac, Linux, and windows. Unfortunately it seems as though using zfs in such a mixed environment is not a good idea any longer unless you like constant support headaches. From what I have read windows will be ok but when Linux and Mac are added to the mix, potentially even if using nfs and not cifs, acls are prone to getting whacked. Please tell me I am missing something as this seems like a terrible idea made with the best of intentions making zfs (and Solaris/oi) unusable in one of the areas it used to shine (and that I had taken for granted after setting it up on older versions before). Sent from my Droid Incredible. "Nikola M." wrote: I am forwarding this to openindiana-discuss@openindiana.org list, with hope of wider audience regarding question. Original Message Message-ID:<4db68e08.9040...@googlemail.com> Date: Tue, 26 Apr 2011 11:19:04 +0200 From: achim...@googlemail.com List-Id: Hi! We are setting up a new file server on an OpenIndiana box (oi_148). The spool is run-in version 28, so the "aclmode" option is gone. The server has to serve files to Linux, OSX and windows. Because of the missing aclmode option, we are getting nuts with the file permissions. I read a whole lot about the problem and the pros and cons of the decision of dropping that option in zfs, but I absolutely read nothing about a solution or work around. The problem is, that gnome's nautilus as well as OSX' finder perform a chmod after writing a file over ifs, causing all ACLs to vanish. If there is no solution, zfs seems to be dead. How do you solve this problem? Achim This has caused me constant headaches, especially as I add rsync to the linux, nfs, samba and windows mix. My current quick and dirty fix is a 15 min. re-apply acl script, which is often enough for my purposes. Since it is all workgroup mode, there isn't any other option for me yet. I have experimented with NFS4 on Centos, which can see acls, but that didn't help at all. For me, it was a matter of choosing the ability to scrub the zpool or have problematic acl's and acl's lost out. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Choice of disks: Will 4kB Blocksize Seagate Barracuda Green ST2000DL003 64MB 2TB work OK in OI148??
On 07/05/2011 12:26 a.m., Hans J. Albertsson wrote: Rather, question is how to deal with the 4kB block size! Choice of disks: Will 4kB Blocksize Seagate Barracuda Green ST2000DL003 64MB 2TB work OK in OI148?? I can confirm, As they "emulate" 512 sectors, yes they will "work", _but_ not as well as Hitachi and some other brand/models, and definitely not recommended with expanders. Also sector alignment may need to be tinkered to improve performance. The major issue is the fixed and very long TLER setting, which will cause driver time-outs, especially under load. Solaris doesn't tolerate this well, and linix isn't too fussed with raid sets of them either. I managed to lock up a server last week with sata bus errors on a triple esata setup with wd20ears disks (raidz LTO5 alternative) with bacula. Mark B. On 2011-05-06 08:34, Hans J. Albertsson wrote: Or are there any gotchas? I intend to put 4 identical ones, as mirrored pairs in ZFS. It's on a SuperMicro XB7Si motherboard, the 4 onboard SATA connectors left after the system rpool's 2 500GB Seagate Barracuda. These will serve up media files as an SMB/Cifs share (iSCSI??) for various media players ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] hotswap strategy
On 26/05/2011 9:13 a.m., Pete Ashdown wrote: I'm having trouble figuring out how to hotswap dead drives on a potential fileserver I am building. I'm using an LSI Internal SATA/SAS 9211-8i& 9211-4i to connect 12 SATA drives on a Supermicro backplane that supports SES-2. I am finding conflicting information as to whether OpenIndiana can light the bay LED of the failed drive on command. If it can, how do I do it? If it can't, how do I identify which bay to swap the drive out of? Right now I've got a mapping of drive serial #'s to bays, but that isn't ideal for a group of admins to execute upon. Also, although the backplane supports SES-2, I'm not seeing /dev/es/ses0. I installed system/io/tests, but I don't have sestopo in that package. The LSI SAS2 hba controllers don't seem to have ses2 support at present. As far as I could tell, the ses functionality isn't present in the firmware or drivers of the sas2 products. see http://kb.lsi.com/KnowledgebaseArticle15874.aspx ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ZFS issues and the choice of platform
On 26/05/2011 9:54 a.m., Pete Ashdown wrote: This gives me some pause as I'm in the process of replacing Nexenta boxes that also had flakey reliability. Is anyone using 148 in an enterprise situation with Supermicro? I have three with 3Gb SAS, of around 40Tb. Two are on oi148, and the last will be updated soon. All are Supermicro X7DWN+ in 3U + 4U jbod, with 6 x 1G and 2 x 10G network ports. Shares are SMB and NFS (3 and 4). These now run the Seagate 2Tb SAS interfaced disks. The disk choice is critical with expanders. The next one, due in about a week, will be sas2 with multipath and the same disks. I've had a good run, averaging two "events" a year per server which required a reboot or in one case, a full OS reinstall. My build recipe is a text install and bash script to add/remove packages and configure networking etc. I use snmp for monitoring, including fault managemnt extension. Mark. On Wed, May 25, 2011 at 10:59:19PM +0200, Roy Sigurd Karlsbakk wrote: Hi all I have a few servers running openindiana 148, and it's been running rather well for some time. Lately, however, we've seen some hichups that may be related to the platform, rather than the hardware. The actual errors have been variable. Some issues were due to some supermicro backplanes that tended to fail, causing drives to report massive i/o errors. But then, the really bad ones are issues where zfs reports bad drives even though iostat report them as good. So far, we haven't lost a pool, it has been sorted out, but I still wonder what happens if I'm gone for a few weeks and something like that happens. The systems where we have had issues, are two 100TB boxes, with some 160TB "raw" storage each, so licensing this with nexentastor will be rather expensive. What would you suggest? Will a solaris express install give us good support when the shit hits the fan? ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] hotswap strategy
On 28/05/2011 3:25 a.m., Pete Ashdown wrote: On 05/27/2011 02:10 AM, Mark wrote: On 26/05/2011 9:13 a.m., Pete Ashdown wrote: I'm having trouble figuring out how to hotswap dead drives on a potential fileserver I am building. I'm using an LSI Internal SATA/SAS 9211-8i& 9211-4i to connect 12 SATA drives on a Supermicro backplane that supports SES-2. I am finding conflicting information as to whether OpenIndiana can light the bay LED of the failed drive on command. If it can, how do I do it? If it can't, how do I identify which bay to swap the drive out of? Right now I've got a mapping of drive serial #'s to bays, but that isn't ideal for a group of admins to execute upon. Also, although the backplane supports SES-2, I'm not seeing /dev/es/ses0. I installed system/io/tests, but I don't have sestopo in that package. The LSI SAS2 hba controllers don't seem to have ses2 support at present. As far as I could tell, the ses functionality isn't present in the firmware or drivers of the sas2 products. see http://kb.lsi.com/KnowledgebaseArticle15874.aspx Any recommendations for a 2U SAS/SATA controller that does have ses2? LSI don't have a HBA that has support. It is only in the raid controllers ie Megaraid. You probably can use a Megaraid card which I think just "passes through" any disk that is not part of a raid set. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] NTPD and PPM issues?
On 28/05/2011 7:32 a.m., Dan Swartzendruber wrote: Andrew Gabriel wrote: It may be the only one which _notices_ it's having this issue. Like it may be the only one which notices if a disk returns a corrupted block. You didn't say what the other OS's are. Sorry, I should have provided more information. VM #1: OpenIndiana. Running ntpd version 4.2.5 - PPM messages seen. VM #2: Ubuntu 10.10. Running ntpd version 4.2.4 - no messages seen. VM #3: FreeBSD 8.2. Running ntpd version 4.2.4 - no message seen. Another point: this is on a quad-core 3.2ghz xeon. Very lightly loaded. I've experienced serious time sync issues with Windows and linux on ESX over the years. Worst was causing AD/kerberos issues, with rapid clock skew causing ntp to move out of capture range and head into oblivion. This is well documented by VMware. I changed ntp settings for windows to increase to 10x the allowable drift, but this cannot be set in OI or linux. My "fix" there was to turn off ntp, and run a cron job every 15 mins to set the clock with ntpdate. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] OpenIndiana-discuss Digest, Vol 10, Issue 60
On 30/05/2011 9:28 p.m., Hans J. Albertsson wrote: Of course: just me being sloppy: I am looking for a card that runs off the fastest possible i/f on the X7BSi MB PCI-Express, absolutely. Just be aware that a bad esata connection can bring down the server. I crashed my oi bacula server a few weeks ago while writing to a 3 x esata disk set. The log was full of sata timeout issues on one port. Message: 4 Date: Sun, 29 May 2011 17:53:25 +0200 (CEST) From: Roy Sigurd Karlsbakk To: Discussion list for OpenIndiana Subject: Re: [OpenIndiana-discuss] A good and reliable SATA PCI interface for a Supermicro X7BSi MB and OI148? Message-ID:<3729368.6.1306684405848.JavaMail.root@zimbra> Content-Type: text/plain; charset=utf-8 Message: 2 Date: Sun, 29 May 2011 12:35:05 +0200 (CEST) From: Roy Sigurd Karlsbakk To: Discussion list for OpenIndiana Subject: Re: [OpenIndiana-discuss] A good and reliable SATA PCI interface for a Supermicro X7BSi MB and OI148? Message-ID:<31113433.0.1306665305717.JavaMail.root@zimbra> Content-Type: text/plain; charset=utf-8 Can anyone suggest a good and reliable eSATA PCI interface for a Supermicro X7BSi MB and OI148? Preferably 4 or 6 eSATA ports To be used in a RaidZ2 config with 8 or 10 2TB relatively cheap disks. 4 connected to the MB SATA II ports, and 4 or 6 on the external. Any particular reason for not using SAS with this? LSI 1068 work well and are affordable if not cheap. They only support 3Gbps SAS, but then, you probably won't need more. The LSI 1068E has external connectors. An LSI 1068 is a chip, not a PCI card. Sure, but there are several cheap boards using it. Btw, are you sure you want a PCI card, not a PCI Express card? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk. -- Message: 5 Date: Sun, 29 May 2011 11:54:23 -0400 From: "Dan Swartzendruber" To: "'Discussion list for OpenIndiana'" Subject: Re: [OpenIndiana-discuss] A good and reliable SATA PCI interface for a Supermicro X7BSi MB and OI148? Message-ID:<6A370FAC738C4A9891687981F59E2794@manticore> Content-Type: text/plain; charset="us-ascii" -Original Message- From: Roy Sigurd Karlsbakk [mailto:r...@karlsbakk.net] Sent: Sunday, May 29, 2011 11:53 AM To: Discussion list for OpenIndiana Subject: Re: [OpenIndiana-discuss] A good and reliable SATA PCI interface for a Supermicro X7BSi MB and OI148? Message: 2 Date: Sun, 29 May 2011 12:35:05 +0200 (CEST) From: Roy Sigurd Karlsbakk To: Discussion list for OpenIndiana Subject: Re: [OpenIndiana-discuss] A good and reliable SATA PCI interface for a Supermicro X7BSi MB and OI148? Message-ID:<31113433.0.1306665305717.JavaMail.root@zimbra> Content-Type: text/plain; charset=utf-8 Can anyone suggest a good and reliable eSATA PCI interface for a Supermicro X7BSi MB and OI148? Preferably 4 or 6 eSATA ports To be used in a RaidZ2 config with 8 or 10 2TB relatively cheap disks. 4 connected to the MB SATA II ports, and 4 or 6 on the external. Any particular reason for not using SAS with this? LSI 1068 work well and are affordable if not cheap. They only support 3Gbps SAS, but then, you probably won't need more. The LSI 1068E has external connectors. An LSI 1068 is a chip, not a PCI card. Sure, but there are several cheap boards using it. Btw, are you sure you want a PCI card, not a PCI Express card? *** Good point. It isn't hard to saturate the PCI bus nowadays... ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] LSI 9211 and multipath
Has anyone tried multipath on a single lsi 9211-8i to a dual port
backplane ?
I have the controler connected to a SM 847E26 and although I see two of
every disk (before and after enabling multipath), it doesn't seem to be
quite working.
The 9211 is using IR mode, so I can get the ses support.
stmsboot -L
stmsboot: No STMS devices have been found
mpathadm list initiator-port
Initiator Port: w500605b0035b6420
Initiator Port: w500605b0035b6420
Initiator Port: iqn.1986-03.com.sun:01:6b3db0fb.4df8134d,402a00ff
format output:
AVAILABLE DISK SELECTIONS:
0. c1t5000C5000A85BA8Ad0 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,4027@7/pci8086,3500@0/pci8086,3510@0/pci1000,3020@0/iport@f/disk@w5000c5000a85ba8a,0
1. c1t5000C5000A85D446d0 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,4027@7/pci8086,3500@0/pci8086,3510@0/pci1000,3020@0/iport@f/disk@w5000c5000a85d446,0
2. c2t5000C5000A85BA89d0 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,4027@7/pci8086,3500@0/pci8086,3510@0/pci1000,3020@0/iport@f0/disk@w5000c5000a85ba89,0
3. c2t5000C5000A85D445d0 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,4027@7/pci8086,3500@0/pci8086,3510@0/pci1000,3020@0/iport@f0/disk@w5000c5000a85d445,0
So there are definitely two paths, via two different adapter ports, to
each of the two disks installed.
I have tried turning on multipath in the controller, but no change.
One thing I'm not sure of is should the target port for a disk be the
same for both paths?
disk, instance #1
name='inquiry-serial-no' type=string items=1 dev=none
value='3LQ3C4489846NZ8H'
name='target-port' type=string items=1
value='w5000c5000a85d446'
disk, instance #3
name='inquiry-serial-no' type=string items=1 dev=none
value='3LQ3C4489846NZ8H'
name='target-port' type=string items=1
value='w5000c5000a85d445'
Mark.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Configure static IP with nwam
On 19/06/2011 11:54 a.m., Serge Fonville wrote: Hi, I recently installed a oi_148 system. By default every interface is configured with DHCP and NWAM I read I can configure NWAM with static IPs and that it would be 'better' that way somehow. Unfortunately I can not find any resource what I need to do in what order to have all four interfaces configured with static IPs. What do I need to do to set this up? Thanks in advance What do I need to do, to set this up Kind regards/met vriendelijke groet, Serge Fonville http://www.sergefonville.nl Convince Google!! They need to add GAL support on Android http://code.google.com/p/android/issues/detail?id=4602 ___ from my build script svcadm disable network/physical:nwam svcadm enable network/physical:default #This is a 10Gb nic, so substitute your dev name for ixgbe0 #mtu change is not required. #10BbE # MTU to 9000 dladm set-linkprop -p mtu=9000 ixgbe0 #static ip ipadm create-addr -T static -a local=10.1.18.19/29 ixgbe0/v4 #don't forget a route - route -p add default 10.1.18.254 #bonding dladm create-aggr -d igb0 -d igb1 0 dladm create-aggr -d igb2 -d igb3 1 #Note: cannot both be on an existing subnet - if one down, both go down. ipadm create-addr -T static -a local=10.1.17.10/24 aggr0/v4 ipadm create-addr -T static -a local=10.1.19.10/24 aggr1/v4 ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Question about drive LEDs
On 19/06/2011 7:49 a.m., Roy Sigurd Karlsbakk wrote: Hi all I have a few machines setup with OI 148, and I can't make the LEDs on the drives work when something goes bad. The chassies are supermicro ones, and work well, normally. Any idea how to make drive LEDs wirk with this setup? Hi Roy, There is no support in the OS, on generic hardware, for this sort of functionality. It "could" be coded in on most systems where ses and ipmi support is present, since much of the functionality is present, but doing this this 'way' outside my comfort zone. LSI 6gb IT mode HBA's don't support ses at present. Cmd line tools to blink led's will probably be available in the next Sol11 release. For compatible lsi sas2 controllers, sas2ircu supports a locate function blink leds. As an aside, I have built a fully functional "7210 Unified Storage Server Clone" running on Supermicro hardware and some customised definitions in the management software to match the new hardware. That setup fully supported drive locator and failure indications, so the generic hardware can do it with the right software. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] multipath on by default ?
I've been battling for a week with my first sas2 implementation and am losing badly. It appears that the oi147 text installer has multipath enabled in the mpt_sas.conf by default. OI installs fine, but the kernel freezes about where the mpt_sas driver message shows up unless I boot in single user mode first, and then to normal. I did try enabling multipath boot, and trying two controllers, but that didn't help. Add in buggy LSI 9211 firmware when using a SM chassis, and it isn't looking good. I'm trying to use IR firmware to get ses but may try changing that to IT mode tomorrow. Has anyone "been there, done that" with this combo ? Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Question about drive LEDs
On 22/06/2011 1:38 a.m., Fred Liu wrote: As an aside, I have built a fully functional "7210 Unified Storage Server Clone" running on Supermicro hardware and some customised definitions in the management software to match the new hardware. That setup fully supported drive locator and failure indications, so the generic hardware can do it with the right software. Mark. Can you shed more lights on it? Thanks. look here. http://stored-on-zfs.blogspot.com ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] Buggy mptsas or multipath or something ...
I have spent more than a week battling my first sas2 storage build. It may be a bug somewhere, but I'm not sure what to point a finger at but I can reproduce it. The goal is about 50Tb of usable storage built mainly with Supermicro kit, with a supported controller. I have build a number of these with 3G sas components and, with the right disks, they work great. Apart from the "standard" issues with sata disks and expanders, and working out what to avoid, things have been pretty good on sas generation 1, but things have moved on to generation 2. It's been deja vu for an IT fossil like me, as it's like going back to the bad old days of fibre channel, where vendors all built gear to the same standard, but none of it would talk to anyone else's. The first encounter was this nice "vision multiplier", 4 devices for the price of one... http://kb.lsi.com/KnowledgebaseArticle16414.aspx Having overcome that hurdle, it was on to the next. With any of my 2Tb Sas disks installed, the oi_147 text installer simply exited to a locked up state when enumerating the disks. So did Solaris 11 text installer, so it is not just oi. Dropping to the shell first and checking out what disks were visible was revealing. cfgadm only showed the two for the os. format saw all four, well sort of. AVAILABLE DISK SELECTIONS: 0. c1t5000C5003434E723d0 255 sec 252> /scsi_vhci/disk/@g5000c5003434e723 Now that's weird, this is multipath. But neither of the os disk's are !! and they are connected to the same controller and backplane. 2. c2t5000C5000A85BA89d0 sec 897> /pci@0,0/pci8086,4027&7/pci8086,3500&0/pci,3510&0/pci1000,3020@0/iport&f0/disk@w5000c5000a85ba89,0 I know that sas2 allows multilane operation between the controller and expanders if the vendor chooses to implement it, so multipath may be needed as it is enabled with the installer mpt_sas.conf, but something isn't quite right here. With only the two os disk plugged in, the installation completed normally, as did the reboot. I then disabled the multipath option from /kernel/drv/mpt_sas.conf and rebooted and things still worked, so I took a deep breath and plugged in the rest of the disks. No crash, and a big bunch of "normal" disks appeared. I disabled the multipath as on one of the many "loops" over several days, I had done this and crashed the server when the disks were added with multipath left at the installer default of enabled. check cfgadm - all listed the same as the OS, and also good in format. I should also add I started this install with two controllers intending to use multipath, but having had so much "fun" even getting to this point, it will stay single path until I have a spare month or two to take it further. Hardware: SM 847E26 chassis, X7DWN+ m/b and 2 x LSI 9211-8i fw P9(patched) in IR mode, 30 x ST32000444SS. So when is multipath the wrong path ? When the installer takes a $ each way. And I don't even have two controllers installed, but there "may" be more than one path, if you are the right device that is. I conclude the mptsas driver will probably require more work as vendor uptake on the many new sas2 features available appear. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Question about drive LEDs
On 23/06/2011 1:25 a.m., Robin Axelsson wrote: On 2011-06-22 11:05, Mark wrote: On 22/06/2011 1:38 a.m., Fred Liu wrote: As an aside, I have built a fully functional "7210 Unified Storage Server Clone" running on Supermicro hardware and some customised definitions in the management software to match the new hardware. That setup fully supported drive locator and failure indications, so the generic hardware can do it with the right software. Mark. Can you shed more lights on it? Thanks. look here. http://stored-on-zfs.blogspot.com That's a very nice web interface! How did you get that on OpenIndiana? It isn't OI, it is a port of the OS from this http://www.oracle.com/us/products/servers-storage/storage/unified-storage/index.html ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Buggy mptsas or multipath or something ...
On 23/06/2011 2:50 a.m., Lucas Van Tol wrote: I saw similar issues to that vision multiplier with any firmware> P7 for the 9211-8i cards and an expander; perhaps you would see better results if you downgraded? Also, I think the system gives multi-path device names even if there is only one path; try mpathadm list lu to see if it is actually doing anything with multiple paths. -Lucas Van Tol No lu's ever appeared, even with two adapters installed, but it did show a port. With two adapters, only one showed up after enabling multipath boot, inspite of it listing two during the activation. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] [zfs-discuss] Question about drive LEDs
On 23/06/2011 6:52 a.m., Pasi Kärkkäinen wrote: On Sat, Jun 18, 2011 at 09:49:44PM +0200, Roy Sigurd Karlsbakk wrote: Hi all I have a few machines setup with OI 148, and I can't make the LEDs on the drives work when something goes bad. The chassies are supermicro ones, and work well, normally. Any idea how to make drive LEDs wirk with this setup? Some questions: - So the Supermicro chassis has SES support? SES can be presented in a number of ways, but basically it is there. It is normally presented to an OS by the Controller connected to the backplane. For SAS, via the sas bus and controller. For Sata, via i2C or GPIO usually to the controller or motherboard. - You're able to see which disk in which chassis slot, by the info from SES? Yes. - Are you able to control the LEDs manually through SES? Yes, via the sas (or scsi) commands understood by the ses device. - Did you configure FMA in any way? For OpenSolaris/OI, it doesn't currently have the appropriate software for non-Sun hardware. With the right hardware, and some work, there are some ways to do it. If you have an LSI 2008 SAS2 controller with IR firmware, this method will work. Storage Module Disk Ready-to–Remove LED Does Not Work Using cfgadm (6946124) http://download.oracle.com/docs/cd/E19452-01/835-0799/v2pn.gkdej.html#scrolltoc I plan to write something to make this easier (but only only with this hardware). Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Question about drive LEDs
On 23/06/2011 7:55 a.m., Gary wrote: On Wed, Jun 22, 2011 at 6:25 AM, Robin Axelsson wrote: That's a very nice web interface! How did you get that on OpenIndiana? Unless I'm mistaken, you can download the VirtualBox image of the simulator here: http://www.oracle.com/webapps/dialogue/dlgpage.jsp?p_ext=Y&p_dlg_id=8588618&src=6870265&Act=7 Yes, and I ported it onto "real" hardware. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Question about drive LEDs
Blinking led's is a "nice to have", but if your server supports ipmi and ses, the fault management report has already located the disk. So I need to replace the disk in ses-enclosure=1/bay=6/disk=0 Apr 15 2010 22:25:27 abbbe612-1131-6add-f5c9-8537ecb46dc7 DISK-8000-0X 100% fault.io.disk.predictive-failure Problem in: hc://:product-id=LSILOGIC-SASX36-A.1:server-id=:chassis-id=50030480005a337f:serial=6 XW15V2S:part=ST32000542AS-ST32000542AS:revision=CC34/ses-enclosure=1/bay=6/disk=0 Affects: dev:///:devid=id1,sd@n5000c50021f4916f//pci@0,0/pci8086,4023@3/pci15d9,a680@0/sd@24, 0 FRU: hc://:product-id=LSILOGIC-SASX36-A.1:server-id=:chassis-id=50030480005a337f:serial=6 XW15V2S:part=ST32000542AS-ST32000542AS:revision=CC34/ses-enclosure=1/bay=6/disk=0 Location: 006 Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Question about drive LEDs
On 24/06/2011 11:09 p.m., Fred Liu wrote: > Mark, > > Can you post the content of your blog in this list? > > Many thanks. > >>> >>> look here. >>> >>> http://stored-on-zfs.blogspot.com >>> In brief, I cloned a bootable SATA disk from the VMware (now discontinued) Sun Unified Storage simulator, and then created the graphics and definitions to match a Supermicro 4U chassis and system board. Everything worked exactly as a real one would, including disk locator led's, disk present/absent graphics etc. and was able to be updated with new firmware. Since it was origionally designed as an generic appliance kit and there was mention of it being available as Software, this was relatively easy to do. It is definitely Solaris based, but has some quite different drivers etc. than OpenSolaris. IPMI is required to make it work. There are quite a few xml files to configure, along with graphics. It is also very easy to crash the management svc with invalid definitions. sample definition: Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Kernel Panic installing openindiana on a HP BL460c G1
On 27/06/2011 7:11 p.m., Johan Guldmyr wrote: On 06/27/2011 10:01 AM, Johan Guldmyr wrote: When I try to install it, it kernel panics at 0%. By adding -k to the grub line I got the the :msgbuf (attached in .zip file - several .jpg files), but I don't understand most of it. If anybody could help me shed some light in this I would appreciate it a lot! Now with the pictures, hope this works: http://imageshack.us/photo/my-images/402/oi151paniccdollarstatus.jpg/ http://img3.imageshack.us/i/oi151panicmsgbuf1.jpg/ http://img5.imageshack.us/i/oi151panicmsgbuf2.jpg/ http://img89.imageshack.us/i/oi151panicmsgbuf3.jpg/ http://img23.imageshack.us/i/oi151panicmsgbuf4.jpg/ http://img16.imageshack.us/i/oi151panicmsgbuf5.jpg/ http://img694.imageshack.us/i/oi151panicmsgbuf6.jpg/ Have you tried deleting the solaris partition from the hard disk (via format) ? It can help prevent detection of an existing, broken filesystem. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ntpd not keeping time in sync
>On 10/07/2011 5:43 a.m., Dan Swartzendruber wrote: No, I think he meant resetting the time in the BIOS of the VM. -Original Message- From: Gary Driggs [mailto:gdri...@gmail.com] Sent: Saturday, July 09, 2011 1:42 PM To: Discussion list for OpenIndiana Subject: Re: [OpenIndiana-discuss] ntpd not keeping time in sync On Jul 9, 2011, at 8:48 AM, Gary Gendel wrote: I suppose this could be true of a virtual machine resetting the time as well. A guest OS should never be allowed to adjust its hosts clock. Sometimes a failing motherboard battery can cause issues but NTP should be correcting them. Have you tried resarting or disabling/enabling the service? If this is a virtual server, then it is "normal" and the drift will depend on physical server load. The physical timers used by the kernel to keep accurate period counts used for local clock don't exist, and the "virtual" replacements don't offer much precision. The host server's cmos clock isn't referenced. This phenomenon is well documented by VMWare, and outcome varies from one OS to the next, depending on how they derive time. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ntpd not keeping time in sync
On 10/07/2011 8:54 a.m., Richard L. Hamilton wrote: AFAIK, at least historically, the hardware battery clock time is expected (without some tweaks, to the extent that a given version allowed those) by Windows to be in local time. Operating systems that keep their internal time in something else (.e.g. Unix and related, where it's supposed to be in GMT, with the TZ environment variable providing an offset to local time as desired) already have to deal with that (and probably with the possibility of it running in GMT instead). It seems to me that a virtualization environment should simulate an RTC clock for the guest, and should simply keep track of the offset between that time and the host's internal time, to be used to supply an initial value when the guest is started. Both virtualization and dual boot get tricky if there are mixed assumptions as to the RTC being in local vs GMT, especially with the addition of daylight saving time, and most particularly if the guest or less common boot environment is active at start or end of DST. It takes some care on the part of all OS and virtualization product producers _and_ the person setting up such a system, to get the whole situation right. It would probably be helpful if the OP provided more details of whether dual boot or virtualization was involved in their situations; and also if someone would write up a good guide to cases where Solaris/OpenSolaris/OpenIndiana was host or guest or primary or alternate boot with various other common OSs, on how to successfully keep the time consistent (unless something other than consistency was intentional!) across all environments. AFAIK, for semi-modern versions of Windows, there are settings that can allow the RTC to run in GMT and still have the OS in local time (with or without DST). I think most other OSs should also be happy with that, or be easily able to be made happy with that. It's what I'd do if I had a multi-boot Intel box that was having issues with getting the time right on some of the OSs. On Jul 9, 2011, at 1:43 PM, Dan Swartzendruber wrote: No, I think he meant resetting the time in the BIOS of the VM. -Original Message- From: Gary Driggs [mailto:gdri...@gmail.com] Sent: Saturday, July 09, 2011 1:42 PM To: Discussion list for OpenIndiana Subject: Re: [OpenIndiana-discuss] ntpd not keeping time in sync On Jul 9, 2011, at 8:48 AM, Gary Gendel wrote: I suppose this could be true of a virtual machine resetting the time as well. A guest OS should never be allowed to adjust its hosts clock. Sometimes a failing motherboard battery can cause issues but NTP should be correcting them. Have you tried resarting or disabling/enabling the service? -Gary D ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss The hardware RTC is only read at boot and sets the initial date/time. The OS will adjust it's internal time from this initial reference and it's timezone, hence different times between cmos rtc and server e.g daylight time. Then the hardware time ticks are counted by the OS to maintain internal time. This is the sole time source until the next boot. The issue is drift of the internal time ticks against the ntp external reference. When this drift exceeds ntp's "capture" range, you get the error message. I have seen this with virtual (VMWare) Windows and Linux as well. VMware also throws in a few "extra's" at vm bootup, just to make life more interesting, but one running, it's up to the OS's method to maintain time. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ntpd not keeping time in sync
On 13/07/2011 8:37 p.m., Jonathan Adams wrote: ntpdate once an hour/half hour in cron?, or grep for the message in a loop and ntpdate if it sees the message? That is a fix I have used for Virtual Linux guests, and it works well, but don't run ntpd as well to avoid the port conflict. The 500ppm ntp range limit is easy to exceed with virtual hardware, since it isn't "real time" orientated. Mark. but that's just the hacker in me ... On 13 July 2011 02:56, Christopher Chan wrote: On Wednesday, July 13, 2011 09:54 AM, Christopher Chan wrote: On Wednesday, July 13, 2011 06:52 AM, Christopher Chan wrote: On Wednesday, July 13, 2011 03:51 AM, Roy Sigurd Karlsbakk wrote: Sigh... ntpdate -u 192.168.9.1 10 Jul 09:01:15 ntpdate[29929]: step time server 192.168.9.1 offset 3317.946738 sec IIRC ntpd won't adjust the time if the difference is too big, measured between the system clock, inherited from the hardware clock, and the ntp servers. I've see this on VMs a few times on Linux - if the guest OS doesn't have a time source compatible, it'll usually drift a lot, possibly enough to make ntpd give up. But then, since you're running on iron (or silicon or something), adjusting the time on bootup with ntpdate should suffice :) I have switched from broadcastclient to specifying 5 time servers. That seems to have got it to behave properly. Spoke too soon...it's off by nine minutes after 12 hours Jul 13 07:42:44 bradsuper1 ntpd[414]: [ID 702911 daemon.notice] frequency error 512 PPM exceeds tolerance 500 PPM Jul 13 08:02:27 bradsuper1 ntpd[414]: [ID 702911 daemon.notice] frequency error 512 PPM exceeds tolerance 500 PPM What causes this?!?! The only thing that has changed is the fact we are in the summer break and we have blooming renovators/builders coming in but they don't start at 7:42am. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] zpool in sorry state
On 14/07/2011 2:44 a.m., Christopher Chan wrote: On Wednesday, July 13, 2011 09:28 PM, Eric Pierce wrote: Is the 9211-8i a different driver than mpt_sas? No it is not. LSI 2008 chip - mpt_sas driver ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss The IT & IR firmware both use the same mpt_sas driver. I'm currently using IR and pass through all the sas disks. There are a few quirks with multi-path and failed drives, but it has been solid so far. I also have a few running mpt and sas disks that are very stable. For Sata disks, very few give good stability, and the model I found to be the best is now different - gone to 4k internal sectors. Mark ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] zpool in sorry state
On 14/07/2011 8:44 a.m., Gregory Youngblood wrote: On Jul 13, 2011, at 1:11 PM, Mark wrote: On 14/07/2011 2:44 a.m., Christopher Chan wrote: On Wednesday, July 13, 2011 09:28 PM, Eric Pierce wrote: Is the 9211-8i a different driver than mpt_sas? No it is not. LSI 2008 chip - mpt_sas driver The IT& IR firmware both use the same mpt_sas driver. I'm currently using IR and pass through all the sas disks. There are a few quirks with multi-path and failed drives, but it has been solid so far. I also have a few running mpt and sas disks that are very stable. For Sata disks, very few give good stability, and the model I found to be the best is now different - gone to 4k internal sectors. The hitachis? They changed the sectors w/o changing part/model/sku? Not yet (waiting for new owners WD to change that!!) , but my favourite Seagate ST3200542AS is now a 4k "green" 4k internal sector drive. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ntpd not keeping time in sync
On 14/07/2011 11:54 a.m., Gary wrote: On Wed, Jul 13, 2011 at 1:04 PM, Mark wrote: The 500ppm ntp range limit is easy to exceed with virtual hardware, since it isn't "real time" orientated. The original poster, Chris Chan, is running it on bare metal hardware. Regardless, I've never had that issue with running NTP in guests hosted in VirtualBox or VMware. If that happened to me in a VM, however, I'd be filing a bug with the VM software vendor. If it is hardware, then check the bios setup for a Spread Spectrum clock setting and try disabling it if it is on. Mark. -Gary ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] can't get into system after hostname change
On 15/07/2011 3:03 p.m., Brett Dikeman wrote: Greetings all, I just changed the hostname on an OpenIndiana machine by changing /etc/hostname.(ifname), /etc/nodename, and rebooting. Networking is down, and worse, I can't login using known-good credentials. On the console was an error about SMF not starting, and on each login attempt, this appears: "Solaris_audit getadrinfo(hobbes) failed[node name or service name not known]: Error 0 Solaris_audit adt_get_local_address failed, no Audit IP address available, faking loopback and error: Network is down Login incorrect" The system has a static IP and is not using directory services. Obviously, this is a big problem...and I'm a bit under the gun to fix it. Suggestions would be most welcome; I have console access, and possibly virtual media access, at the moment. Much appreciated, Brett Did you try booting to single user mode ? Then update /etc/hosts with it's new ip. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hardware fault, drive or controller?
On 16/07/2011 9:18 a.m., Matt Connolly wrote: Hi all, following my crash and boot failure during the week, I left my oi machine off for a day or so to think about what it had done wrong. After turning it back on and booting off the live oi-148 cd, I was able to import my 2 pools, and did a scrub of the boot pool. There were a few checksum errors, but everything recovered correctly. I then rebooted back into the installation on the HDs and it was looking good. I let it run doing a scrub of my data pool after reimporting it, which was running fine for many hours. However, it locked up again. The system was still running, (I could use firefox and make network connections) but anything that interacted with the data pool hung. `zpool status` hung, for example, and could not be terminated with Ctrl+C. I shut the machine down and saw lots of errors in /var/adm/messages like: Jul 14 07:10:01 vault genunix: [ID 859416 kern.info] ghd_timer_newstate: HBA reset failed hba 0x ff01ca571e40 gcmdp 0xff01db863a00 gtgtp 0xff01cc10f6c0 Jul 14 07:10:01 vault scsi: [ID 107833 kern.warning] WARNING: /pci@0,0/pci8086,2448@1e/pci-ide@0 /ide@1 (ata1): Jul 14 07:10:01 vault timeout: HBA reset, target=1 lun=0 Jul 14 07:11:01 vault genunix: [ID 859416 kern.info] ghd_timer_newstate: HBA reset failed hba 0x ff01ca571e40 gcmdp 0xff01db863a00 gtgtp 0xff01cc10f6c0 Jul 14 07:11:01 vault scsi: [ID 107833 kern.warning] WARNING: /pci@0,0/pci8086,2448@1e/pci-ide@0 /ide@1 (ata1): Jul 14 07:11:01 vault timeout: HBA reset, target=1 lun=0 I've since completed a scrub on both pools with no errors, so I'm fairly sure that it's not a dead hard drive. Does this sound like a motherboard/controller failure?? Any other thoughts? Thanks, Matt Don't forget to check/replace the cables as well if it is sata. Sata tends to run better in native sata mode too. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] exporting rpool with system running
On 02/08/2011 12:14 a.m., Harry Putnam wrote: Roy Sigurd Karlsbakk writes: AFAIK, exporting an rpool while running on it, is quite impossible. Why would you want to export it? So use livecd then? The reasons to do this have been discussed here at least twice. Thats why I didn't include that information. Thinking about it again after your prompt though I see I should have included at least a brief outline of why. So briefly put: The hardware is giving up the ghost. The machine will start and run for maybe 10-15 minutes and then stops with no log errors evident. Rather than worry with the old outdated equipment I plan to build up a newer hardware setup. Install openindiana on a pair of smallish disks for the new (mirrored) rpool and then import all the old zpools including the oldrpool (renamed) which, unfortunately, also has data I'd like to keep on it. Oldrpool is also on a mirrored pair of disks. So I thought to leave oldrpool exported to facilitate the above approach. Install the disks containing oldrpool and import it using the new setup, then dump all the old rpool OS related stuff and keep the wanted data on a renamed zpool. Better ideas are welcome. Just install the disks and boot. I have found it to be very portable. I have done an rpool Sata boot disk from VM Workstation raw disk 32 bit, to older AMD64 dual cpu, to modern Intel 5420 dual cpu and back again. It just booted without issues. Nic does change, but that's easy to handle. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] exporting rpool with system running
On 06/08/2011 1:34 a.m., Harry Putnam wrote: Reginald Beardsley writes: [ reordered by ed -hp] --- On Mon, 8/1/11, Harry Putnam wrote: From: Harry Putnam Subject: Re: [OpenIndiana-discuss] exporting rpool with system running To: openindiana-discuss@openindiana.org Date: Monday, August 1, 2011, 7:14 AM Roy Sigurd Karlsbakk writes: AFAIK, exporting an rpool while running on it, is quite impossible. Why would you want to export it? Detach the mirror disk from the old rpool. Does this need to be done before migrating the disks to the new setup? That is, should I run the old hardware... do `zpool detach ...' on the boot discs (the other zpools are already exported) and then shutdown and migrate the discs? I haven't done this step in the past with my set of testing disks. With Sata, it always seems to boot successfully, but if mirrored, not usually with the mirror disk intact, so that does need to be added back. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] [zfs-discuss] Question about WD drives with Super Micro systems
On 07/08/2011 5:45 a.m., Roy Sigurd Karlsbakk wrote: Might this be the SATA drives taking too long to reallocate bad sectors? This is a common problem "desktop" drives have, they will stop and basically focus on reallocating the bad sector as long as it takes, which causes the raid setup to time out the operation and flag the drive as failed. The "enterprise" sata drives, typically the same as the high performing desktop drive, only they have a short timeout on how long they are allowed to try and reallocate a bad sector so they don't hit the failed drive timeout. Some drive firmwares, such as older WD blacks if memory serves, had the ability to be forced to behave like the enterprise drive, but WD updated the firmware so this is longer possible. This is why you see SATA drives that typically have almost identical specs, but one will be $69 and the other $139 - the former is a "desktop" model while the latter is an "enterprise" or "raid" specific model. I believe it's called different things by different brands: TLER, ERC, and CCTL (?). I doubt this is about the lack of TLER et al. Some, or most, of the drives ditched by ZFS have shown to be quite good indeed. I guess this is a WD vs Intel SAS expanders issue Vennlige hilsener / Best regards roy -- The other overlooked thing is the different topology ie. native sata against sas translated sata and then sas expander as well. I have had seagate ST3000641's just refuse to run with expanders, but work diecty connected to the sas controller. It's funny how all this reminds me of fibre channel behaviour in it's early days. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] [zfs-discuss] Question about WD drives with Super Micro systems
On 07/08/2011 10:40 p.m., Roy Sigurd Karlsbakk wrote: The other overlooked thing is the different topology ie. native sata against sas translated sata and then sas expander as well. I have had seagate ST3000641's just refuse to run with expanders, but work diecty connected to the sas controller. It's funny how all this reminds me of fibre channel behaviour in it's early days. Then why do Hitachi and Seagate drives work flawlessly for me? We have some slow Seagate drives (ST32000542AS) in two chassises, and those just work. Another server, using Hitachi HDS721010CLA332 drives also just works - one dead drive during burn-in, and that was all. Western drives on direct attach works, but with SAS expanders, we get I/O errors (as reported by zpool and iostat) on high load. My conclusion is that WD don't really want desktop drives to work on anything but a direct sata connection port. The WD desktop disk's firmware appears to be designed to fail on hardware raid and sas. I've tested a range of them and found that early versions of drives worked well, but firmware changes/updates to later versions of the same disk now fail, usually under load. I suppose for WD, there had to be a defining boundary between Desktop and Enterprise, and Hardware Raid and Expanders are firmly pushed to the enterprise side. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] BAD WD drives - defective by design?
On 30/08/2011 6:39 p.m., Roy Sigurd Karlsbakk wrote: I somewhat doubt it's a driver thing, since it happens only with WD drives, and it happens with both the mpt and mpt_sas drivers. roy Some sata drives just don't handle the sas-sata emulation combined with sas expanders. I have found early WD EADS drives with the firmware updated are fine with expanders, and later ones (different version and firmware) aren't. To make life more fun, don't trust a drive model to be any assistance in selecting what works. All manufacturers seem to change the specs and keep the model designation the same. The 24 x Seagate 6Gb 2Tb SATA disks I had that wouldn't work with expanders are now a solid FC target box via multiple Marvel 8 port PCI-X cards direct to each disk. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] 6 GB SATA drives on SAS HBAs
On 31/08/2011 10:12 a.m., Rich wrote: FWIW, crossflashing from IR to IT or vice-versa requires that you boot into DOS with their flasher and firmware, nuke the existing firmware, and replace it - hopefully not rebooting in the middle, because your card would be a brick if you did. :) Of course, if you did this, and it still refused to flash once you'd removed the firmware entirely, I don't have any thoughts. c.f. http://kb.lsi.com/KnowledgebaseArticle16266.aspx Some Supermicro cards and onboard controllers with this chipset require a power cycle to change between IR to IT firmware, and the firmware tools can flash it in that state. I've successfully recovered a few with bad firmware as well i.e. no heartbeat You also need to watch the 1068 version (usually B1, B2 or B3) The firmware must match. I have successfully used LSI code to update the Supermicro 8023C to IR code when Supermicro couldn't supply it for a B2 chip. Speed negotiation over expanders can be "interesting". On new builds I use the controller diags to check the negotiated link speed is both correct and stable between power cycles. Some disks, especially 6G Seagate desktop, just don't negotiate well on 3G topology, and similar issues around 3G disks on 6G topology. Also some expander code versions are better than others. I've also RMA'ed Supermicro backplanes with issues as well. So you can be sure it is a minefield out there. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] Comstar FC target
Is anyone using the Comstar FC target ? I have two experimental ones running oi148. One is using HP FC disk trays and disks on one port of the hba, and the target on the other. With three esx4i hosts connected, it runs quite well. The other is a "Thumper" clone with 20 x 2Tb sata disks. It goes like a rocket - about 3 x faster than a large ds4500 on sequential writes which is it's main function, but one of the esx4i hosts seems to cause it to reboot under load occasionally. I'm looking for ideas to trace the cause, as there is nothing logged prior to the reboot. Both these setups are single fc path and 4Gb Fibre. I'm planning to start changing things with the client HBA firmware, since the three hosts attached have different models of hba. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Best way to map SAS drives to physical positions
On 07/10/2011 1:52 p.m., James C. McPherson wrote: On 7/10/11 10:41 AM, Christopher Chan wrote: On Wednesday, October 05, 2011 02:03 AM, Alasdair Lumsden wrote: Hi Jeppe, On 3 Oct 2011, at 14:19, Jeppe Toustrup wrote: The only way I have found which I can find the order of the drives in the JBOD along with the above names, without pulling out each drive and see which one becomes unavailable, is within /etc/path_to_inst, where I find the following lines: Have you tried: /usr/lib/fm/fmd/fmtopo -V ? It works with SES compatible JBOD enclosures and I can confirm it works with the LSI JBODs we use. Thanks Alasdair for this! I get enclosure id and bay id with this command. But boy, it sure does need lots of parsing. Or you could look at the headers in usr/include/scsi and write a utility that takes a topology snapshot, then walks and parses the entries therein. fmtopo or ipmi topo requires ipmi to be available. With IPMI, yy fmadm output does report what Slot (Bay's numbering starts at 0, Slots at 1) a faulty disk is in. Alternatively, if your enclosure AND controller support ses, you can also use gs2_utils and sg_ses. The latest version has a --locate option which is missing from the repository version. I plan to compile the latest version and test it on a few storage combinations to see if the magic happens. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Mixing zfs raids and mirrors
On 18/10/2011 4:51 a.m., James Carlson wrote: Gabriele Bulfon wrote: Hi, I was wondering what (if any) problems I may encounter by mixing different zfs strategies on one pool, with all equal disk sizes. Examples: - pool1: raidz1 a0 a1 a2 mirror a3 a4 - pool2: raidz1 b0 b1 b2 raidz1 b3 b4 b5 mirror b6 b7 - pool3: mirror c0 c1 mirror c2 c3 raidz c4 c5 c6 I'm just trying to balance pools performance and maximum space... Last but not least: - what if pool3 is actually the rpool (where system just occupies few GBytes), where I'll add volumes with quotas to share? Should I keep rpool separate absolutely? (I just don't want to miss many GB on an rpool that will just stay as it is, on the boot disks). Last I checked, it wasn't possible to boot off of RAID-Z; only mirrors were supported for boot. As for the other part, the man page says: Virtual devices cannot be nested, so a mirror or raidz vir- tual device can only contain files or disks. Mirrors of mir- rors (or other combinations) are not allowed. If you're looking to boost RAID-Z performance, I'd suggest adding more disks. RAID-Z will effectively stripe the data across the available devices, parallelizing the I/O operations. To get better error tolerance, use raidz2 or raidz3. zfs will also complain if your choices are likely to impact performance e.g adding a 3 disk raidz to an existing 5 disk raidz. Performance is optimal when build from the start, since data is evenly spread across the available space. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Where can I find a full Walk thru setting up nfs4 from scratch
On 14/11/2011 1:11 p.m., Harry Putnam wrote: Running openindiana b 151a as vm guest on host win7 Where can I find an up to date detailed description of setting up nfs4 from scratch on a home lan zfs server with linux nfs4 clients. I've turned up loads of stuff googling, way too much to paw through it all. I'm running openindiana and have posted a similar request on that group. Some of the directions I've found insist on setting up nis. Do I really need nis to be setup to share fs with nfs on a home lan? This is a home lan with no real internet address on the inner network, just a homemade 192.168.1.0/24 setup. With a domain name of `local.lan'. I'm not sure how that effects how I setup nfs4. I need the most basic guidance on this... I do not have a working knowledge of nfs. Hopefully there is some url online for real greenhorns that describes how to do this in detail. When I start to edit /etc/default/nfs as described in some of the directions I've found I see: # Moved to SMF. Use sharectl(1M) to manage NFS properties. So at least several of the guides I've run across are apparently out of date and insist on editing /etc/default/nfs. Hard to tell what guides hold water for modern solaris and offshoots. If you can wait a few days, I can give you working configs for nfs4 on Centos to OI, since I have a few running that config at work. I seem to recall that only a few settings are needed to make it all work. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] setting up nfs4 from scratch
NFS 4 setup you need to configure your domain Linux /etc/idmapd.conf [General] Verbosity = 0 Pipefs-Directory = /var/lib/nfs/rpc_pipefs Domain = mydomain.local [Mapping] Nobody-User = nobody Nobody-Group = nobody [Translation] Method = nsswitch - Opensolaris/OI make sure services are running svcadm enable nfs/status svcadm enable nfs/server svcadm enable nfs/nlockmgr svcadm enable nfs/mapid configure Domain sharectl set -p nfsmapid_domain=drg.local nfs The nfs access then needs to be set on the zfs filesystem zfs -o sharenfs=rw=@192.168.1.0/24,root=@192.168.1.0/24, \ anon=0 datapool/mydata mount from linux mount -t nfs4 -o rw 10.1.17.10:/datapool/mydata /mnt/mydata nfs3 does not require the domain settings. I use both nfs3 and nfs4, but GID/UID issues and ACLS can be tricky, especially if you run cifs on the same file system as I do. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] zfs replace controller
On 16/11/2011 11:07 p.m., Matt Connolly wrote: Hi, I've just upgraded my SATA card from a Si3114 which only used the pci-ide driver to a Si3124 which uses the si3124 driver (and is recognised as sata by `cfgadm` etc). I was hoping that I could replace the card and have oi magically find the partitions on the same drives connected to my new controller. My setup has two pools: rpool - mirrored x 3, boot zpool - raidz x3 for data Once I got my machine to booting, getting "zpool" across was easy. Simply `zpool export zpool` then `zpool import zpool`, and hey presto, all works. The boot "rpool" was a bit trickier though. I thought I could do this: root@vault:/# /sbin/zpool replace rpool c4d1s0 c5t3d0s0 invalid vdev specification use '-f' to override the following errors: /dev/dsk/c5t3d0s0 is part of active ZFS pool rpool. Please see zpool(1M). root@vault:/# /sbin/zpool replace -f rpool c4d1s0 c5t3d0s0 invalid vdev specification the following errors must be manually repaired: /dev/dsk/c5t3d0s0 is part of active ZFS pool rpool. Please see zpool(1M). root@vault:/# But I could only get the other drives on the mirror pool by doing a `zpool detach` followed by `zpool attach` which wants to resilver the entire drive. Is this the only way to migrate a boot pool drive from one controller to another? Or can it be "moved" (replaced) without requiring the resilver? In my experience, the boot (rpool) mirror will always break. I don't usually export the boot device, only any other zpools. I have moved between supported sata and sas controllers as well. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] setting up nfs4 from scratch
On 17/11/2011 3:40 a.m., Harry Putnam wrote: Mark writes: [...] Thanks for the very complete instructions. No problem Harry, glad to help. On my linux distro [debian wheezy] I see this in /etc/idmapd.conf # set your own domain here, if id differs from FQDN minus hostname # Domain = localdomain On Centos I set this to the fqdn, but both ends must match. You may also need /etc/hosts entries for the other server. And since `hostname -f (-f means show fqdn) shows my full hostname.local.domain I guess that can stay commented. nfs3 does not require the domain settings. I use both nfs3 and nfs4, but GID/UID issues and ACLS can be tricky, especially if you run cifs on the same file system as I do. So are you saying that even with the settings you posted... you still have trouble with windows boxes over of cifs? Or do you mean your posted settings will avoid that happenstance? The issues are mainly around ACL's, but in my case the files rsync through multiple servers before landing, and most don't support ACL's. I'm trying just to run nfs4 so maybe it will not effect me. nfs4 understands ACL's, but nfs3 doesn't. Oh, and what chmod cmd do you use on any shares to be shared with windows platforms? In the past, for cifs, I've used: chmod -R A=everyone@:full_set:fd:allow When security isn't an issue, then this will be easier. Files created from windows will probably end up showing different ACL's as Windows uses different defaults, and nfs3 and 4 also will produce different results, since one knows ACL's and one doesn't. I just run a cron job to bash them back into what I want. I'd go with a simple approach that works for you. I haven't seen any major performance difference between cifs and nfs. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] 3x LSI 9211-8i controllers + direct attach backplane -- appear as on controller
On 18/11/2011 12:51 a.m., Rich wrote: On Wed, Nov 16, 2011 at 11:52 PM, Eric D. Mudama wrote: On Wed, Nov 16 at 13:47, Bill Sommerfeld wrote: On 11/16/11 13:27, James C. McPherson wrote: (and apart from "I don't understand it therefore it must be bad" I don't know why you really would) I don't like mpxio disk names -- they make me work too hard and create fear of doing the wrong thing. Long hex strings are much harder to distinguish than short strings and it's much easier to get confused between two devices. humans are reasonably good at dealing with dense sets of small integers, and not quite so good at dealing with sparse sets of 64-bit and 128-bit values. are c2t5000C5002C68468Bd0 and c2t5000C5002C68468Bd0 the same or different? what about c2t5000C5002C68203Bd0 and c2t5000C5002C689ABFd0? are c2t5d0s0 and c2t6d0s0 the same or different? how about sd5a and sd5a? how much time did each of those comparisons take you? how quickly were you sure of your answer? The important part to me is that the 000C5002C68468B above is typically printed on the label of the drive. Put a little sticker on your sled with the contained WWN, and you won't make the mistake of grabbing the wrong drive from the array again. (They need a support group for that) SES means you can blink the drive LED and, with a convenient mapping method, not have this issue again. :) - Rich A simple "locate" application is still elusive, inspite of SES. And LSI 6G SAS controllers lack SES support in the IT firmware, so I run mine in IR mode (with no raid defined). A single bad disk (SATA, SAS or FC) still stops the server dead while it retries. ZFS really could do with kernel and driver tuning. With a raid system, ignore a read error and just correct and deliver the data, and then write it back which in most cases triggers a disk bad sector replacement and the problem is sorted quickly. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] /dev/bmc on SuperMicro boards
On 19/11/2011 8:25 a.m., Jason J. W. Williams wrote: Hi, I'm trying to load the BMC driver from Solaris 11 on my OpenIndiana b151 install. It's a Supermicro X8DTL server board. The BMC driver is loaded into the correct place I believe: -rwxr-xr-x 1 root sys21920 Nov 17 00:44 /kernel/drv/bmc -rwxr-xr-x 1 root sys 177 Nov 17 00:44 /kernel/drv/bmc.conf -rwxr-xr-x 1 root sys34216 Nov 17 00:44 /kernel/drv/amd64/bmc However, it doesn't seem to come up automatically on boot, and manually running modload on amd64/bmc reports: can't load module: No such device or address Has anyone had luck getting the BMC driver to work with SM boards? I have run the bmc from the last nvs release on OpenSolaris on a Supermicro and it works fine. I used the SIMSO+ on a X7DWN. I haven't tried the newer onboard Realtec chipsets yet. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] 3x LSI 9211-8i controllers + direct attach backplane -- appear as on controller
On 19/11/2011 4:52 p.m., Rich wrote: I bet you $50 you can use sas2ircu on the IT firmware, particularly with the locate command. [Don't take the bet; if you do, I'll link you to a private github repo with some code that does it, and my PayPal account.] - Rich Hi Rich, Locate is possible, but the "application" isn't in the OS yet. It was a while ago when I had was no ses device with IT f/w P8. I did find an LSI kb article saying it wasn't implemented yet. Mind you, I thought I was having multipath issues with this gremlin at the time. http://kb.lsi.com/KnowledgebaseArticle16414.aspx I see there is now a p11 out, so it may need an upgrade and try again. I see that the latest sg3_utils now has a locate cmd, so it may be time to compile a newer one again and try it. Mark. On Fri, Nov 18, 2011 at 6:29 PM, Jason J. W. Williams wrote: It would be nice if disk manufacturers offered firmware flavors with quick fail timeouts/reduced retries on failed read/write ops. Along the lines of the firmware loads they give array OEMs. -J Sent via iPhone Is your email Premiere? On Nov 18, 2011, at 15:48, Mark wrote: On 18/11/2011 12:51 a.m., Rich wrote: On Wed, Nov 16, 2011 at 11:52 PM, Eric D. Mudama wrote: On Wed, Nov 16 at 13:47, Bill Sommerfeld wrote: On 11/16/11 13:27, James C. McPherson wrote: (and apart from "I don't understand it therefore it must be bad" I don't know why you really would) I don't like mpxio disk names -- they make me work too hard and create fear of doing the wrong thing. Long hex strings are much harder to distinguish than short strings and it's much easier to get confused between two devices. humans are reasonably good at dealing with dense sets of small integers, and not quite so good at dealing with sparse sets of 64-bit and 128-bit values. are c2t5000C5002C68468Bd0 and c2t5000C5002C68468Bd0 the same or different? what about c2t5000C5002C68203Bd0 and c2t5000C5002C689ABFd0? are c2t5d0s0 and c2t6d0s0 the same or different? how about sd5a and sd5a? how much time did each of those comparisons take you? how quickly were you sure of your answer? The important part to me is that the 000C5002C68468B above is typically printed on the label of the drive. Put a little sticker on your sled with the contained WWN, and you won't make the mistake of grabbing the wrong drive from the array again. (They need a support group for that) SES means you can blink the drive LED and, with a convenient mapping method, not have this issue again. :) - Rich A simple "locate" application is still elusive, inspite of SES. And LSI 6G SAS controllers lack SES support in the IT firmware, so I run mine in IR mode (with no raid defined). A single bad disk (SATA, SAS or FC) still stops the server dead while it retries. ZFS really could do with kernel and driver tuning. With a raid system, ignore a read error and just correct and deliver the data, and then write it back which in most cases triggers a disk bad sector replacement and the problem is sorted quickly. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Fileserver performance with log and cache devices?
On 3/12/2011 9:34 a.m., Per Sjoholm wrote: On 12/02/2011 07:44 PM, Gary Mills wrote: On Thu, Dec 01, 2011 at 08:59:48AM -0600, Gary Mills wrote: I'm in the process of setting up a fileserver running oi_151a. How can I determine the performance improvement from adding log and cache devices? I'm using filebench with the fileserver personality. Is that my best choice? So far, I've tried filebench directly on the server and also on an NFS client. What statistics should I be watching? So, can nobody answer my questions? http://constantin.glez.de/blog/2011/02/frequently-asked-questions-about-flash-memory-ssds-and-zfs # How Do I Know if an SSD Will Really Help My Write Performance? <http://constantin.glez.de/blog/2011/02/frequently-asked-questions-about-flash-memory-ssds-and-zfs#write> # How Do I Know if an SSD Will Really Help My Read Performance? <http://constantin.glez.de/blog/2011/02/frequently-asked-questions-about-flash-memory-ssds-and-zfs#read> To begin, I'm using a single 1 TB SAS disk on the server, without log or cache devices. I intend to add an Intel 311 SSD for the log and Intel 320 SSDs for the cache. I'd like to ensure that these improve NFS performance. I'd like CIFS and zfs send performance to be reasonable as well. The simple answer is, they can help, but I doubt you will see much improvement with a single disk, since that is the major bottleneck. The standard disk i/o performance guidelines like spindle count and rpm still apply. Read cache is only useful if the same file or data is being read multiple times, since the cache needs to be populated. For writes, if the server has plenty of memory and a zpool with an optimal spindle count is as fast as an ssd, then it won't give as much benefit, although it does have the benefit of being non-volatile. I seem to recall mention that the dedicated zil is only used if it's needed. I have great performance from a FC target using 3 x SAT2-MV8 and 20 x 2Tb sata disks without an ssd zil. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Will an 82575EB based Supermicro AOC-SG-i2 ethernet work OK in b151a?
On 15/12/2011 6:35 a.m., James Carlson wrote: Hans J. Albertsson wrote: Subject says it all. I suppose so, at least one shouldn't need to do anything beyond addig a line in /etc/driver_aliases, right? The line for 8086,10a7 should already be there; it's served by "igb". I have used both the 2 & 4 port non-uio 1Gbit and two port 10G non-uio Supermicro cards with no changes, since early OI releases up to 151. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Solaris 11 source code leaked?
On 20/12/2011 6:29 a.m., Gregory Youngblood wrote: This seems potentially bad all the way around to me. Hopefully this turns out to be a real, officially blessed release and not a leak. It doesn't look to be official, since it does contain some closed source code but is mostly CDDL and even contains an OPENSOLARIS.LICENSE ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] How will this change affect any future source from Oracle ?
This looks to be a major change to how ON/net is being built. http://blogs.oracle.com/ali/entry/much_ado_about_nothing_stub Will this be a show stopper when integrating any future source released by Oracle ? Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] PS2 keyboard problems
Does anyone else have constant problems getting a ps2 keyboard to work ? I have struck it on at least 4 different servers from as many vendors, and not one has a working ps2 keyboard once OI (or any other flavoured Solaris) has booted. All my OI servers now run ps2 to usb adapters. I finally have found time to make a working build environment, so it may be time to dig deeper. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] PS2 keyboard problems
On 9/01/2012 6:08 a.m., Open Indiana wrote: Whta do you mean? If you disconnect a PS2 device on a running server it will never reconnect without booting. That is at least what I found out more than once on all kind of servers (windows2003/2008/solaris etc) No, I mean the keyboard works perfectly up until the OS boots, and from then on it ceases to work. This is the case on my current TYAN S2915, Supermicro X7DWN+ and the same issue has been encountered on other motherboards as well. The keyboard used makes no difference. Mark. -Original Message- From: Mark [mailto:mark0...@gmail.com] Sent: zondag 8 januari 2012 8:07 To: openindiana-discuss@openindiana.org Subject: [OpenIndiana-discuss] PS2 keyboard problems Does anyone else have constant problems getting a ps2 keyboard to work ? I have struck it on at least 4 different servers from as many vendors, and not one has a working ps2 keyboard once OI (or any other flavoured Solaris) has booted. All my OI servers now run ps2 to usb adapters. I finally have found time to make a working build environment, so it may be time to dig deeper. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Solaris 11 11/11 Opensource part1 and part2
On 2/02/2012 9:45 p.m., Open Indiana wrote: "their pricing is just way out through the stratosphere..." I could Licence Windows Server 2008 R2 X64 on my OI box for around $700 p/a (2 processor) Oracle want $1488 for EACH cpu, but since it isn't certified, won't offer support. definitely off this planet. Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] Some SNMP settings
Here is some of my SNMP monitoring that someone may find useful.
Monitoring OI with SNMP
Fault Management
echo 'dlmod sunFM /usr/lib/fm/amd64/libfmd_snmp.so.1' >>
/etc/net-snmp/snmp/snmpd.conf
snmp oid: iso.org.dod.internet.private.enterprises.sun.prod.195.1.4.0
( 1.3.6.1.4.1.42.2.195.1.4.0 )
Logs
(oid here is for the first logmatch entry)
snmpd.conf setting
echo 'logmatch dmesg /var/adm/dmesg 600 WARNING ERROR' >>
/etc/net-snmp/snmp/snmpd.conf
snmp oid:
iso.org.dod.internet.private.enterprises.ucdavis.logMatch.logMatchTable.logMatchEntry.logMatchGlobalCounter.1
( 1.3.6.1.4.1.2021.16.2.1.5.1 )
Custom scripts
monitor zpool space to work around snmp disk size limit.
This returns text based on use, but can be altered to suit your
requirements. One entry per zpool.
The actual oid will depend on the number of extended options in use.
snmpd.conf setting
echo 'extend datapool /usr/bin/bash /usr/bin/zpools.shdatapool' \
>> /etc/net-snmp/snmp/snmpd.conf
snmp oid:
iso.org.dod.internet.private.enterprises.netSnmp.netSnmpObjects.nsExtensions.nsExtendObjects.nsExtendOutput2Table.nsExtendOutput2Entry.nsExtendOutLine.8.100.114.103.118.97.117.108.116.1
(
1.3.6.1.4.1.8072.1.3.2.4.1.2.8.100.114.103.118.97.117.108.116.1 )
-
#!/bin/bash
#zpools.sh
#
export PATH=/usr/bin:/usr/sbin:/sbin
export LD_LIBRARY_PATH=/usr/lib
diskused=`zpool list -H -o capacity ${1} | sed -e 's/%//g' `
#echo $diskused
if [ $diskused -ge 84 ]
then echo "WARNING"
exit 1
fi
if [ $diskused -gt 90 ]
then echo "CRITICAL"
exit 1
fi
if [ $diskused -lt 84 ]
then echo "OK"
fi
-
Mark.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Dell R510 SAS2 Backplane with SAS2008 (PERC H200)
On 8/04/2012 3:26 p.m., Sergei wrote: Rich acm.jhu.edu> writes: ...what FW modifications did they make? - Rich They definitely made some as you can't flash Dell drive with Seagate/WD firmware. Or get support from Seagate/WD for Dell-branded drive. That is specific to the firmware protection settings, and is "standard" for most OEM'd drives. You probably can flash a Seagate with Dell firmware though. Specifically in this case drive reports support for this power management command but fails to respond correctly when Solaris sends such command. I'm pretty sure I was told that Seagate OEM drive works fine while we were having these power issues with identical drive purchased from Dell. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] disconnected drives, how to avoid in the future?
On 13/04/2012 9:42 a.m., Rich wrote: Those patches aren't yet in OI/IL mainline, as of when I looked today. Regarding when they'll be usable, either in mainline or by fetching them yourself... 17:33< PMT> ping Triskelios - I don't suppose you have your pending patches to mpt_sas (per http://blogs.everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/) laying around somewhere easily grabbable? 17:34<@Triskelios> not at the moment, should land on our public repo on bitbucket sometime soon - Rich Just a word of caution to other hackers like me. I was experimenting with shorter drive time-outs and destroyed the zpool while testing different time-outs on some sata disks (on 8 port Marvel controller). The drives in the pool (20) all dropped like flies when the io usage increased, and I had to redo it from scratch. In my experience, drive issues are often a bad block, seen by a jump in iostat errors, and that can occasionally impact performance well before the disk actually fails. I now check iostat regularly, and spare out a disk with issues before the failure becomes permanent, although it does not always avoid a big impact as some drives can fail during the re-silver process. Running a full read/write using format is usually enough to fix the bad block permanently. If not, then it's RMA'd. Mark. On Thu, Apr 12, 2012 at 1:30 PM, Karl Rossing wrote: I'm running into this issue with disconnected drives on snv_134. Would upgrading to oi_151a2 have the updated mpt_sas drive as noted on http://blogs.everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/ "Update (New): These timeouts don’t do squat because mpt_sas doesn’t honour the timeouts. This was recently uncovered by Nexenta and a patch to fix it is about to hit Illumos shortly. I’ll post when it does. Another patch is in progress which will further improve how mpt_sas handles failed drives. Thanks to Albert Lee for his work on them - you, sir, rock!" Karl On 01/10/2012 10:48 AM, Martin Frost wrote: >From: Jason Matthews >Date: Tue, 10 Jan 2012 08:26:08 -0800 > > >you can adjust the disk timeouts in solaris. Here's an article on how to do that, although it ends with the author adding this comment "However in testing with failing harddrives (on mpt_sas anyway), we see that the sd timeouts are completely ignored so my entire post above is moot!" http://blogs.everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/ I haven't tested this, so does it work or not (in OpenIndiana)? Martin >there are two schools of thought here: > >1) accomodate the extremely long timeouts of cinsumer drives and >let the drive decide whether to report an error back (fail itself >out) > >2) set the time outs very narrowly and be aggressive in letting zfs >fail out disks. > >i generally go with option 2. > >Sent from Jasons' hand held > >On Jan 10, 2012, at 7:13 AM, Maurilio Longo wrote: > >>Geoff, >> >>I've hit this problem several times in the past, with OpenSolaris >>and then with OpenIndiana. >> >>There are, to my knowledge, no available solutions, it is so by >>design! >> >>If a disk stops responding the pool waits until after it responds >>again (sometimes pulling it out of its slot and then reinserting >>the disk causes a reset of the link and it starts working again). >> >>I was not able to assess what happens if I set failmode to continue. >> >>I think it could be no better since you still cannot write to the pool. >> >>This is IMHO the biggest problem of ZFS, in that I cannot >>instruct it to stop using a failed device if it has some level of >>redundancy still available. >> >>Wait is OK only if an entire vdev stops responding, not if a disk >>in a vdev with redundancy has problems either fatal or >>transitory. >> >>Best regards. >> >>Maurilio. >> >> >>PS. Using server grade disks (those with TLER) makes it possibile >>to overcome this problem for transitory errors. >> >> >>Geoff Nordli wrote: >> >>>Part of my concern is why one disk would have completely brought >>>down the system. I have seen this come up on the list before, >
Re: [OpenIndiana-discuss] no external connectivity on link
And don't forget to configure or disable the firewall. That's what my last issue with lack of connectivity turned out to be. It may be on and blocking by default. On 2/05/2012 6:13 p.m., Richard Elling wrote: On May 1, 2012, at 8:41 PM, Tim Dunphy wrote: hello list I have attempted to enable link aggregation on my oi 151 box using the command dladm create-aggr -d e1000g0 -d e1000g1 1 then I plumbed it with an address of 192.168.1.200 and echoed 192.168.1.1> defaultrouter I noticed that my /etc directory did not have the traditional solaris hostname.e1000g0 and hostname.e1000g1 files representing my links that you would typically expect. But I created a file with the command echo 192.168.1.200> hostname.aggr1 ifconfig -a shows the aggregated link as well and all indications are that it's fine dladm show-aggr shows the link aggregation and I can ping it..however I cannot ping anything else on or off the network (like 192.168.1.1 or yahoo.com) can someone offer a tip on what I'm doing wrong and how to resolve the issue? Typical error is an incorrect netmask. Default netmask for a class-B network is 255.255.0.0. once I get this working I plan to enable link trunking on my switch to take advantage of the faster speeds possible with this setup. Are you prepared for no-improvement? Ethernet link aggregation often disappoints people looking for more performance using IP. -- richard ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
There are two issues. The first is correct partition alignment, the second ashift value. In "theory", I haven't tested this yet, manually creating the slices with a start position to sector 64 and using slices instead of whole disks for the zpool devices, and creating with an ashift of 12 may produce the desired outcome. I have used 4k disks (wd20ears) in 3 and 4 disk raidz pools, but they are used for archiving, so just have data dumped to them. Few issues on Sata ports, but dodgy on SAS. Mark On 1/05/2012 8:23 a.m., Peter Wood wrote: I'm building a storage server with Dell MD1000 DAS and I just bought 30 drives with 4K sectors. One of the reasons I selected the "new" 4K sector is so I can easily find replacement drives 2-3 years from now when they start failing. Looks like this was a huge mistake. I'm fine if the drives report 512B sectors and work in slower legacy mode as long as they work reliable but seems that this may not be the case. On top of that my internal drives that make the rpool have 512B sectors so I'm not sure how workarounds will effect this pool. Is it fair to say that if one uses 4K drives he will run into alignment issue sooner or later? I'm really puzzled what to do here. Should I try to replace the drives with 512B ones before the storage goes life? Any thoughts? Thank you Peter On Mon, Apr 30, 2012 at 7:54 AM, Richard Elling< richard.ell...@richardelling.com> wrote: On Apr 29, 2012, at 7:38 PM, Gordon Ross wrote: On Sun, Apr 29, 2012 at 8:46 PM, Richard Elling wrote: On Apr 29, 2012, at 11:45 AM, George Wilson wrote: [...] Speaking of 4K sectors, I've taken a slightly different approach that fixes this outside of ZFS. The idea is to allow sd to override the physical-block-size which ZFS will pick up. The way this works is you can specify the Vendor/Product id in sd.conf. Here's an example: sd-config-list = "NETAPP LUN, "physical-block-size:4096"; This is the preferred solution and there are several implementations running around in various stages of test/release/acceptance. I look forward to getting this upstream :-) -- richard Providing a work-around in "sd" is great. We should do that, at least. But is it sufficient? What happens if I replace a mirrored drive with 512 byte sectors with one having 4k sectors? What if I want to plan ahead for that? Maybe in only some of my ZFS pools but not all? It would seem that a pool-level override for "ashift" might also be useful. ashift is set for the top-level vdev at creation time. So you have to override prior to creation of the mirror. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Hitachi 4 TB disks + HP Microserver + OI 151 + ZFS
On 3/05/2012 7:07 a.m., Gordon Ross wrote: On Wed, May 2, 2012 at 2:47 PM, Richard Elling wrote: On May 2, 2012, at 12:25 AM, Mark wrote: There are two issues. The first is correct partition alignment, the second ashift value. In "theory", I haven't tested this yet, manually creating the slices with a start position to sector 64 and using slices instead of whole disks for the zpool devices, and creating with an ashift of 12 may produce the desired outcome. Default starting offset for slice 0 is sector 256 for 512-byte sector disks. You shouldn't have to manually touch these unless you are running an ancient (circa 2006-2007) version of Solaris where the default offset was 34. Yes, but be warned: the format command will try to make fdisk partitions that are aligned on "cyninder" boundaries, which are usually odd numbers of sectors. I have a fix for that here: http://yalms.org/cr/zfs-blksize/ (the first three files. ignore the zfs part for now) Gordon One day this low level disk stuff will get into this millenium. I see my CDC Wren IV 94171-327 museum exhibit still rates a mention :) Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] [discuss] SPARC and Illumos
One last thing: I bought a Dual 1.5GHz USIV Uniboard for V490 which is located in Phoenix, but the seller strictly refuses to ship overseas. Could smb. please help me and have it forwarded to Berlin? This would be cool. But let's first get the DVD out. I guess the seller can wait 2 more days. Later later ... I just bought a V890 for NZD 72.00 !! but that is for another task. Glad I didn't have to ship it !! But I do have a V440 waiting patiently for an OI install. (4 x CPU, 8Gb mem 4 x 72Gb Hdd) It can be made available for remote testing. Mark. tnx. %martin ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] Sata Port Multiplier
I have been trying to get the si3124 driver working with port multipliers using the si3826 chipset. The driver source suggests port multipliers are supported. The card's are seen, driver loads, but no disks. The controller ports with multipliers connected show as failed. The server is the Backblaze Pod 3.0 system using Syba PCI Express SATA II 4-Port RAID Controller Card SY-PEX40008 and AC-SAN-5PMBP 5 Bay Port Multiplier Backplane (Sil 3826 Chipset). No issues when I tried running Nas4Free, but FreeBSD doesn't support the application I want to use for the storage. I did try turning on debugging, but as a beginner at driver debugging I'm not sure I achived the desired result and got no useful info from it. Anyone have experience with Sata Port Multipliers and can share a few pointers ? Mark. ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Status of compilers in use for OI
On 13/07/2013 9:13 a.m., Jim Klimov wrote: On 2013-07-12 21:52, Bryan N Iotti wrote: Thank you all for helping make this clearer. So, in order to compile something using the same tools that are used to build /stable, I would use pkg:/developer/sunstudio12u1 while for hipster I'd use GCC 4.6.3? Unfortunately, no. There was a specific post-release patched version of Sun Studio which is not redistributable, so basically only those who have downloaded it sometime back in 2010 can still use it. It is not available from Oracle website (unlike the base SS12 or SS12u1 which are not it), as well as the OI-bundled package is not it. The package in the OI does provide some other tools needed for the build, but not the specific known-to-work-unlike-others version of the compiler. Would this be a file called sunstudio12-patched-ii-2009Sep-sol-x86.tar.bz2 ? //Jim ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] [zfs] problem on my zpool
One suggestion for the future for better reliability. Avoid using WDC WD20EARS, they aren't great, and hopeless on SAS controllers. Avoid mixing disk brands or models in a zpool. The different behaviour can trigger issues. On 23/10/2013 7:46 p.m., Clement BRIZARD wrote: I cleared the "degraded" disk. we will see what happens in 131hours pool: nas state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Wed Oct 23 08:25:56 2013 2.23G scanned out of 22.2T at 48.6M/s, 133h22m to go 6.10M resilvered, 0.01% done config: NAME STATE READ WRITE CKSUM CAPProduct nas ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 c8t50024E9004993E6Ed0p0 ONLINE 0 0 0 2 TB SAMSUNG HD204UI c8t50024E92062E7524d0ONLINE 0 0 0 2 TB SAMSUNG HD204UI c8t50024E900495BE84d0p0 ONLINE 0 0 0 2 TB SAMSUNG HD204UI c8t50014EE25A5EEC23d0p0 ONLINE 0 0 0 2 TB WDC WD20EARS-00M c8t50024E9003F03980d0p0 ONLINE 0 0 0 2 TB SAMSUNG HD204UI c8t50014EE2B0D3EFC8d0ONLINE 0 0 0 2 TB WDC WD20EARX-00P c8t50014EE6561DDB4Cd0p0 ONLINE 0 0 0 2 TB WDC WD20EARS-00M c8t50024E9003F03A09d0p0 ONLINE 0 0 0 2 TB SAMSUNG HD204UI raidz1-1 ONLINE 0 0 0 c50t8d0 ONLINE 0 0 0 (resilvering) 2 TB ST2000DL004 HD20 c2d0 ONLINE 0 0 0 (resilvering) 2 TB c1d0 ONLINE 0 0 0 (resilvering) 2 TB c50t11d0 ONLINE 0 0 0 2 TB SAMSUNG HD204UI c50t10d0 ONLINE 0 0 0 (resilvering) 2 TB SAMSUNG HD204UI Le 23/10/2013 08:43, Clement BRIZARD a écrit : I woke up this morning and so you're messages, unfortunately I had to reboot, the server completely froze. Now I have that : pool: nas state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Wed Oct 23 08:19:42 2013 5.81G scanned out of 22.2T at 49.2M/s, 131h43m to go 15.6M resilvered, 0.03% done config: NAME STATE READ WRITE CKSUM nas DEGRADED 0 0 0 raidz1-0 DEGRADED 0 0 0 c8t50024E9004993E6Ed0p0 ONLINE 0 0 0 c8t50024E92062E7524d0ONLINE 0 0 0 c8t50024E900495BE84d0p0 ONLINE 0 0 0 c8t50014EE25A5EEC23d0p0 ONLINE 0 0 0 c8t50024E9003F03980d0p0 ONLINE 0 0 0 c8t50014EE2B0D3EFC8d0ONLINE 0 0 0 c8t50014EE6561DDB4Cd0p0 DEGRADED 0 0 0 too many errors c8t50024E9003F03A09d0p0 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 c50t8d0 ONLINE 0 0 0 (resilvering) c2d0 ONLINE 0 0 0 (resilvering) c1d0 ONLINE 0 0 0 (resilvering) c50t11d0 ONLINE 0 0 0 c50t10d0 ONLINE 0 0 0 (resilvering) Le 23/10/2013 08:00, Jason Matthews a écrit : first, dont reboot. if you do you might not be able remount the pool. the data you see is from the disks that are functioning. listing the files and copying complete files are two different things. if you dont have a backup you may need to copy whatever partial data you can from the broken pool. now let's start by getting the disks back in good shape. clear the degraded disk zpool clear c8t50014EE6561DDB4Cd0p0 reseat the missing disks in the hopes they come back then clear them check cfgadm -al and make sure they are connected and configured when you reseat them check the messages (or dmesg) to see if the system notices the re-insertion. if it does see the disk installed clear the disks in the pool in effort to bring the pool back to an operational state. Sent from Jasons' hand held On Oct 22, 2013, at 5:04 PM, Clement BRIZARD wrote: Hello everybody, I have a problem with my pool, I had some slowdowns lately on my nfs share of my zfs pool. A weekly scrub began and is still running but it worries me, it currently returne that pool: nas state: UNAVAI
Re: [OpenIndiana-discuss] No boot from iso on HP xw8600
It could be that once booted, it didn't find the dvd drive again. On 20/07/2014 5:35 p.m., Harry Putnam wrote: Setup: Server and Desktop ISO, attempting to boot an HP xw8600 A 2x Xeon (5470) with 32 GB ram and 2 1tb hdd. I've attempted booting an HP xw8600 from both server and desktop (freshly downloaded and burned) isos. In either case I come to one of those offers to drop me into maintenance mode and requests uid and passwd, or press Control + d. Pressing the `Control + d' option results in this message: Requesting System maintenance mode (see /lib/svc/share/README for more information) Console login service[s] cannot run And back comes the offer to enter a uid and passwd or press Control + d for maintenance mode. Using the stock uid - passwd, root - openindiana, gets me into maintenance mode with these messages: July 20 00:46:40 su, `su root' succeeded for root on /dev/console -bash /usr/sbin/quota - no such file or directory openindiana (powered by illumos) SunOS 5.11 oi 151_a8 July 2013 -bash /bin/mail - no such file or directory -bash /usr/bin/hostname - no such file or directory root # If I run `ls -al' I see a listing of /root (.bashrc and .profile) followed by /usr/bin/hostname - no such file or directory... Other basic commands seem to work, such as pwd etc. But trying something like `format' fails miserably with: -bash - format command not found. -bash - /usr/bin/hostname - no such file or directory. Where is the `format' command usually hiding. /bin/, /usr/bin, /sbin, /usr/sbin, all seem to be present and accounted for... but no `format'. `format' is present at /usr/sbin/format on an oi vm I have running, but apparently is not included on this `Live CD' ISO. Is there something I can do here to help the boot. Or, is it likely to be something else causing the trouble? PS - both ISO's work fine when tried on another host and installing into a vm. So at least any thoughts of corruption or the like should be laid to rest. ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] booting HP xw8600 from ISO
On 21/07/2014 7:11 p.m., Harry Putnam wrote: Setup: Hardware: HP xw8600 2x Xeon (5470) 3.3 Ghz - 32 GB ram (goal was to get past whatever prevents oi iso from working when hdw is in default settings.) I've already posted a lament about not being able to boot this hardware from install ISO and instead being dropped into a maintenance mode where key commands such as `format' are not available. I didn't know enough to try to `fix' whatever was the problem from the maintenance mode. I resorted to diddling around with the HP xw8600 bios. I discovered that changing the sata emulation mode at (on the bios initial screen) Storage > Device configuration << NOT HERE >> storage options << under this heading>> [...] Sata emulation [...] That option offers three ways to setup the sata controller It is defaulted to `RAID+AHCI'. But on that setting or trusty ISO will not boot into a usable situation. Dropping instead into a lame maintenance mode. So, turning that option off by selecting either of the other two (below) [...] [Note: copy past from HP xw8600 manual -ed HP] , | SATA Emulation—Sets the SATA emulation mode with the following options: | |RAID + AHCI–both the RAID and AHCI OPROMs execute. This emulation |mode is the default and offers the best performance and most |functionality. | |Separate IDE Controller–offers standard SATA supports (four ports |only). | |Combined IDE Controller–makes the SATA controller look like an IDE |controller and offers best IDE compatibility (two ports only). ` [...] I have some misgivings about either of the other two but discovered that the installation ISO will boot normally under either. I've currently set it to `Separate IDE Controller' but could use some advice as to what is best to do here. You have done what most of us do, tried the available options in the appropriate area. I'll continue the install assuming, that bios setting can be changed if necessary without having to install all over again. Hopefully someone experienced with this server (HP xw8600) will be able to advise me about this item. This issue is seen if the dvd drive isn't working after the kernel is running. I have seen it many times. The boot loader gets the kernel and base drivers and part of the OS into memory, and that usually works. Then it needs the kernel and drivers to recognize the sata controller and drive to mount and access the rest of the system files, which is where it failed. The 'cure" is to try all bios settings affecting the dvd drive, which is what you have successfully done. I have in the past resorted to IDE drives, or booting from USB drives, which can be even more problematic to try to resolve the problem. So well done, you have made it a few rungs up the experience ladder. Mark. ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] booting HP xw8600 from ISO
On 23/07/2014 5:38 a.m., Harry Putnam wrote: Mark writes: On 21/07/2014 7:11 p.m., Harry Putnam wrote: Setup: Hardware: HP xw8600 2x Xeon (5470) 3.3 Ghz - 32 GB ram (goal was to get past whatever prevents oi iso from working when hdw is in default settings.) I've already posted a lament about not being able to boot this hardware from install ISO and instead being dropped into a maintenance mode where key commands such as `format' are not available. I didn't know enough to try to `fix' whatever was the problem from the maintenance mode. I resorted to diddling around with the HP xw8600 bios. I discovered that changing the sata emulation mode at (on the bios initial screen) Storage > Device configuration << NOT HERE >> storage options << under this heading>> [...] Sata emulation [...] That option offers three ways to setup the sata controller It is defaulted to `RAID+AHCI'. But on that setting or trusty ISO will not boot into a usable situation. Dropping instead into a lame maintenance mode. So, turning that option off by selecting either of the other two (below) [...] [Note: copy past from HP xw8600 manual -ed HP] , | SATA Emulation—Sets the SATA emulation mode with the following options: | |RAID + AHCI–both the RAID and AHCI OPROMs execute. This emulation |mode is the default and offers the best performance and most |functionality. | |Separate IDE Controller–offers standard SATA supports (four ports |only). | |Combined IDE Controller–makes the SATA controller look like an IDE |controller and offers best IDE compatibility (two ports only). ` [...] I have some misgivings about either of the other two but discovered that the installation ISO will boot normally under either. I've currently set it to `Separate IDE Controller' but could use some advice as to what is best to do here. You have done what most of us do, tried the available options in the appropriate area. I'll continue the install assuming, that bios setting can be changed if necessary without having to install all over again. Hopefully someone experienced with this server (HP xw8600) will be able to advise me about this item. This issue is seen if the dvd drive isn't working after the kernel is running. I have seen it many times. The boot loader gets the kernel and base drivers and part of the OS into memory, and that usually works. Then it needs the kernel and drivers to recognize the sata controller and drive to mount and access the rest of the system files, which is where it failed. The 'cure" is to try all bios settings affecting the dvd drive, which is what you have successfully done. I have in the past resorted to IDE drives, or booting from USB drives, which can be even more problematic to try to resolve the problem. So well done, you have made it a few rungs up the experience ladder. Hah... and so I have. But is there any reason to prefer one of the settings that work over the other. Either: Separate IDE Controller–offers standard SATA supports (four ports only). (The one I used) Or: Combined IDE Controller–makes the SATA controller look like an IDE controller and offers best IDE compatibility (two ports only). For example: Does the second one above leave more usable sata ports? I'd usually pick "native" sata over emulated, and use ide emulation only when required as a work-around if "native" has issues, not that the help explanation here makes it that clear In my experience, the "native" sata performance is usually better. So you have made the choice I would have. ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] Comstar Fibre Channel Target
Does anyone have experience with Comstar as a Fibre Channel Target ? I build one some years ago, which ran really well, but my latest effort is disappointing. Throughput is abysmal, only reaching about 20 Mbytes/sec depending on block size. Zpool is raidz with 30 x 4Tb SAS disk attached to a 6G SAS IT controller. Local disk throughput reaches over 400 Mbytes/sec. The Initiator is Windows 2012 with Qlogic 8G adapter, and Qlogic 4G at the OI end, with straight cable. It works, no errors I can find, but performance just sucks. In desperation, I'm about to try swapping the Qlogic for an Emulex HBA. Can anyone offer suggestions on identifying possible causes ? Mark. ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Comstar Fibre Channel Target
Thanks Liam, no ssd's. Did you change any hba bios or qlt parameters ? Unfortuntely the only Emulex adapter I have is unsupported - an LPe1150. I tried a different "brand" of 4G QLogic adapter. After adding an alias for it to even be seen, the performance was the same. Around 200Mb/sec read and under 30 for write. I do have access to an 8Gb one I could try next. The luns will eventually be for Commvault storage, with a 64k block size. Trying different zfs block sizes hasn't made a significant difference to the Windows throughput. Mark. On 4/09/2014 9:31 p.m., Liam Slusser wrote: I have a few ZFS servers (OI and OmniOS) that are fibre Channel targets. I use the qLogic 2562 8g dualport FC HBA card in all of them with great success. One of my systems is similar to yours, with 2 x (12 x 4T SAS) attached via a LSI 9207-8e SAS HBA. I am able to saturate both 8gb fibre channel connections with a single host. My target is a Sun/Oracle T4 system running Oracle Solaris 10 also with a qLogic 2562. I generally use a small block size (8k) since I only use these servers for an Oracle RDBMS 12c database. I use the Oracle ASM filesystem on the target side. I've never tried to mount to a Windows server before with my setup. You didn't mention if you had a SSD for ZIL, but you might want to try disabling the ZIL and seeing if that helps your performance. You can also monitor what the ZIL is doing with a great little program written by Richard Elling. http://www.richardelling.com/Home/scripts-and-programs-1/zilstat With this you can monitor the ZIL writes - it should help to figure out if that is your problem or not. Good luck! thanks, liam On Thu, Sep 4, 2014 at 1:29 AM, Mark wrote: Does anyone have experience with Comstar as a Fibre Channel Target ? I build one some years ago, which ran really well, but my latest effort is disappointing. Throughput is abysmal, only reaching about 20 Mbytes/sec depending on block size. Zpool is raidz with 30 x 4Tb SAS disk attached to a 6G SAS IT controller. Local disk throughput reaches over 400 Mbytes/sec. The Initiator is Windows 2012 with Qlogic 8G adapter, and Qlogic 4G at the OI end, with straight cable. It works, no errors I can find, but performance just sucks. In desperation, I'm about to try swapping the Qlogic for an Emulex HBA. Can anyone offer suggestions on identifying possible causes ? Mark. ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Comstar Fibre Channel Target
No switch, just a direct cable at present, so point to point. On 6/09/2014 5:55 a.m., Liam Slusser wrote: Mark - No I didn't change any of the HBA bios or QLT parameters. Are you going into a fibre channel switch? thanks, liam On Fri, Sep 5, 2014 at 1:06 AM, Mark wrote: Thanks Liam, no ssd's. Did you change any hba bios or qlt parameters ? Unfortuntely the only Emulex adapter I have is unsupported - an LPe1150. I tried a different "brand" of 4G QLogic adapter. After adding an alias for it to even be seen, the performance was the same. Around 200Mb/sec read and under 30 for write. I do have access to an 8Gb one I could try next. The luns will eventually be for Commvault storage, with a 64k block size. Trying different zfs block sizes hasn't made a significant difference to the Windows throughput. Mark. On 4/09/2014 9:31 p.m., Liam Slusser wrote: I have a few ZFS servers (OI and OmniOS) that are fibre Channel targets. I use the qLogic 2562 8g dualport FC HBA card in all of them with great success. One of my systems is similar to yours, with 2 x (12 x 4T SAS) attached via a LSI 9207-8e SAS HBA. I am able to saturate both 8gb fibre channel connections with a single host. My target is a Sun/Oracle T4 system running Oracle Solaris 10 also with a qLogic 2562. I generally use a small block size (8k) since I only use these servers for an Oracle RDBMS 12c database. I use the Oracle ASM filesystem on the target side. I've never tried to mount to a Windows server before with my setup. You didn't mention if you had a SSD for ZIL, but you might want to try disabling the ZIL and seeing if that helps your performance. You can also monitor what the ZIL is doing with a great little program written by Richard Elling. http://www.richardelling.com/Home/scripts-and-programs-1/zilstat With this you can monitor the ZIL writes - it should help to figure out if that is your problem or not. Good luck! thanks, liam On Thu, Sep 4, 2014 at 1:29 AM, Mark wrote: Does anyone have experience with Comstar as a Fibre Channel Target ? I build one some years ago, which ran really well, but my latest effort is disappointing. Throughput is abysmal, only reaching about 20 Mbytes/sec depending on block size. Zpool is raidz with 30 x 4Tb SAS disk attached to a 6G SAS IT controller. Local disk throughput reaches over 400 Mbytes/sec. The Initiator is Windows 2012 with Qlogic 8G adapter, and Qlogic 4G at the OI end, with straight cable. It works, no errors I can find, but performance just sucks. In desperation, I'm about to try swapping the Qlogic for an Emulex HBA. Can anyone offer suggestions on identifying possible causes ? Mark. ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Comstar Fibre Channel Target
I've added a switch, but that made things worse. I now see frequent link loss under any load similar to issues discussed here about a year ago. I'm going to flag fibre targets as a dead end, and explore smb or maybe iscsi on 10G ethernet instead. Thanks for the input. Mark. On 6/09/2014 5:55 a.m., Liam Slusser wrote: Mark - No I didn't change any of the HBA bios or QLT parameters. Are you going into a fibre channel switch? thanks, liam On Fri, Sep 5, 2014 at 1:06 AM, Mark wrote: Thanks Liam, no ssd's. Did you change any hba bios or qlt parameters ? Unfortuntely the only Emulex adapter I have is unsupported - an LPe1150. I tried a different "brand" of 4G QLogic adapter. After adding an alias for it to even be seen, the performance was the same. Around 200Mb/sec read and under 30 for write. I do have access to an 8Gb one I could try next. The luns will eventually be for Commvault storage, with a 64k block size. Trying different zfs block sizes hasn't made a significant difference to the Windows throughput. Mark. On 4/09/2014 9:31 p.m., Liam Slusser wrote: I have a few ZFS servers (OI and OmniOS) that are fibre Channel targets. I use the qLogic 2562 8g dualport FC HBA card in all of them with great success. One of my systems is similar to yours, with 2 x (12 x 4T SAS) attached via a LSI 9207-8e SAS HBA. I am able to saturate both 8gb fibre channel connections with a single host. My target is a Sun/Oracle T4 system running Oracle Solaris 10 also with a qLogic 2562. I generally use a small block size (8k) since I only use these servers for an Oracle RDBMS 12c database. I use the Oracle ASM filesystem on the target side. I've never tried to mount to a Windows server before with my setup. You didn't mention if you had a SSD for ZIL, but you might want to try disabling the ZIL and seeing if that helps your performance. You can also monitor what the ZIL is doing with a great little program written by Richard Elling. http://www.richardelling.com/Home/scripts-and-programs-1/zilstat With this you can monitor the ZIL writes - it should help to figure out if that is your problem or not. Good luck! thanks, liam On Thu, Sep 4, 2014 at 1:29 AM, Mark wrote: Does anyone have experience with Comstar as a Fibre Channel Target ? I build one some years ago, which ran really well, but my latest effort is disappointing. Throughput is abysmal, only reaching about 20 Mbytes/sec depending on block size. Zpool is raidz with 30 x 4Tb SAS disk attached to a 6G SAS IT controller. Local disk throughput reaches over 400 Mbytes/sec. The Initiator is Windows 2012 with Qlogic 8G adapter, and Qlogic 4G at the OI end, with straight cable. It works, no errors I can find, but performance just sucks. In desperation, I'm about to try swapping the Qlogic for an Emulex HBA. Can anyone offer suggestions on identifying possible causes ? Mark. ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] nfs oi server - linux clients
I've used linux (centos) nfs4 to OI servers for a few years with better results than nfs3. You do need to set the linux clients up carefully. These are my notes: NFS 4 setup Linux /etc/idmapd.conf [General] Verbosity = 0 Pipefs-Directory = /var/lib/nfs/rpc_pipefs Domain = mydomain.local [Mapping] Nobody-User = nobody Nobody-Group = nobody [Translation] Method = nsswitch == OpenIndiana svcadm enable nfs/status svcadm enable nfs/server svcadm enable nfs/nlockmgr svcadm enable nfs/mapid sharectl set -p nfsmapid_domain=mydomain.local nfs Then set the appropriate nfs permissions on the zfs file system. (I'll need to find the details on this bit if you need it) Mark. On 17/09/2014 1:39 a.m., Gregory Youngblood wrote: I don't know if things have changed but a couple of years ago I had to force Linux to v3 or things would hang or otherwise not work reliably. Sadly I don't recall the details only the lesson to force v3 on Linux clients. Something about the Linux v4 implementation at the time only worked Linux to Linux but not Linux client to Solaris server. Hopefully I am out of date and that's been fixed by now. Greg -- Original message--From: Hugh McIntyreDate: Tue, Sep 16, 2014 1:29 AMTo: openindiana-discuss@openindiana.org;Subject:Re: [OpenIndiana-discuss] nfs oi server - linux clientsYou don't even need NIS or LDAP. Plain /etc/passwd works fine, either by making sure the necessary user/uid mappings and passwd files are the same on all systems (if using NFS v2/v3) or not even bothering with the uid's matching if using NFSv4.(Non-matching uid's is kind of the point of the NFSv4 idmap complexity).Hugh.On 09/16/2014 12:11 AM, Alex Smith (K4RNT) wrote:> I used NIS when I was doing this, while I was beta testing Solaris 9 and> had a Linux client to work with, and that managed to work pretty well,> given I didn't have any connectivity issues between the hosts.>> I know that solution is kinda deprecated, but it's pretty complicated to> set up LDAP comparatively.>> " 'With the first link, the chain is forged. The first speech censured, the> first thought forbidden, the first freedom denied , chains us all> irrevocably.' Those words were uttered by Judge Aaron Satie as wisdom and> warning... The first time any man's freedom is trodden on, we’re all> damaged." - Jean-Luc Picard, quoting Judge Aaron Satie, Star Trek: TNG> episode "The Drumhead"> - Alex Smith> - Huntsville, Alabama metropolitan area USA>> On Tue, Sep 16, 2014 at 1:07 AM, Hugh McIntyre > wrote:>>>>> Hi Harry,>>>> It's possible you have somehow mounted the filesystem locally with noexec>> (unlikely, but you can check with "mount | grep /projects/dv" and make sure>> noexec is not in the options).>>>> But at a guess, it's more likely you may have the wrong username mapping>> since NFSv4 may need configuration for this.>>>> The easiest way to check the user mapping is:>>>> 1. Create a directory on the server with permissions 777 (wide open)>> 2. On the client, "touch somefile">> 3. Then check on both the server and client which user/group the file is>> created as. Do not proceed until the usernames match.>>> If you have a user/group mismatch, then the fix depends on which version>> of NFS you are running. In "traditional" NFS (v3 or v2), the client just>> sends the numeric uid/gid over the wire, and the assumption is that the>> server has the same passwd/group file mapping. In your case though you>> seem to be using nfs4, which works differently.>>>> In NFS v4 (the new default) the configuration is more complex. NFSv4 uses>> names instead of numbers (so you don't need the same UID on all boxes), but>> the complexity is that there's a "nfsmapid" service on Solaris that>> translates from NFS username to local uid/names. This relies on a>> nfsmapid_domain and if this is misconfigured, you get access problems.>> Similarly, rpc.idmapd.conf on Linux.>>>> For the Solaris/Illumos end, Oracle has some info at>> http://docs.oracle.com/cd/E23824_01/html/821-1462/nfsmapid-1m.html but>> the summary for the Solaris end is:>>>> 1. You can specify the NFSMAPID_DOMAIN parameter in nfs(4) or using>> sharectl.>> 2. Or specify the _nfsv4idmapdomain DNS resource record. This is what I>> do since I have a local DNS server and then it works for all hosts.>> 3. Or if neither of these, Solaris will attempt to auto-determine the>> domain based on the domainname command, which may or may not giv
[OpenIndiana-discuss] First cluster after 20 years
I'm putting together a home lab / cluster for computational chemistry as a private individual. I'm thinking OpenIndiana on the head node (for a desktop) and OmniOS on the compute nodes. Are there any downsides to this that I should be mindful of? Thanks! ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org https://openindiana.org/mailman/listinfo/openindiana-discuss
[OpenIndiana-discuss] X apps in lx zone?
Is it possible to run X applications from within an lx zone? If so, would it work for the global zone to run the X server and the lx zone to provide plasma/KDE? ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org https://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] sound juicer doesn't propose mp3 as output?
I hate to say it; but, as a last ditch effort, you could also install the CSW repository under /opt from either Blastwave or OpenCSW. "Hate to say" because the CSW repository can have a fair bit of duplication of existing packages; that, are dependencies to what you may be installing. Also, these repositories use the old PKG format, and are not IPS. Hmmm, just checking BlastWave, it looks like they're introducing a new, stripped-down, more maintained package set around the /opt/bw directory. ( https://www.blastwave.org/wiki/display/BWSTACK/BWTree+Release+Announcement) But, Blastwave is still providing a CSW repository, so if you're going to try one of these, make sure to create a backup/alternate boot environment, so you can revert to a clean-state, and/or revert to another CSW repository. The CSW repository will make additions and modifications to files in your existing system configuration directories. (ie. /etc/opt/csw & /var/opt/csw); so a clean boot environment, is probably the best way to manage your evaluation between implementing the two CSW repositories. Of course, compiling from source would be required, if none of the sources of potential packages identified in this thread work-out for you. Regards, -- .\\ark ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Moving an OI install to a new hard drive?
Hi Dan, Generally speaking, new systems should have new installations done on them to deploy a customized driver set specific to the target system. However, if you're looking at using the same 'rpool' on new hardware, I will trust that you have sorted-through potential hardware / driver issues. If so, you could also try the simple process of mirroring the existing disk, to the new disk, in the old machine. Then just install the new disk in the new machine. The only limitations to this process are: - As mentioned, the hardware has to be very similar to the original system, so that device names aren't changed, and video works, etc. - The new disk has to be the same size as, or larger than, the original. - There is some debate as to whole disk installations, versus partition-based installations. (Reference: http://darkstar-solaris.blogspot.com/2008/09/zfs-root-mirror.html 1. Use "format" and your BIOS to identify the disks in the machine, before installing the new disk. (Let's assume it's on c4t0d0) 2. Power-off; install the new disk; and, power-up again. 3. Re-run "format" to identify the new disk location. (Let's assume c4t1d0) 4. While in "format", actually format the "new" disk 100% as type "solaris 2". 5. Install the old disk geometry descriptors, into the new disk, by exporting them from the old and into the new. # prtvtoc /dev/rdsk/c4t0d0s0 | pfexec fmthard -s - /dev/rdsk/c4t1d0s0 6. Add the new hard-disk to ZFS mirror of the rpool. pfexec zpool attach -f rpool c4t0d0s0 c4t1d0s0 (Wait for mirror process to finish silvering by checking with "zpool status".) 7. Install the GRUB boot-loader to the new hard disk. pfexec installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c4t1d0s0 8. Test that the new hard drive, is a valid stand-in boot device by shutting-down; swapping the connectors on the hard drives; and, booting back-up. 9. If the new hard drive passes the boot test, you should generally be fine. Shutdown and swap the hard drive connections back-to their original position. 10. Pull the hard-drive and install it in the new system. Try booting. If the hardware (ie. video, hard drive controllers, etc.) is similar enough, you should be good-to-go! If it doesn't boot, you may need a rescue disk to go in and change the device references; and, re-install GRUB, etc. >From here, you should only have dangling missing devices on both systems, which should be easy to clean-up (ie. remove), or at least disable. Heck, leave the device reference around for when you're ready to permanently mirror root! Try testing this procedure-out; but, be aware that virtual machines present identical hardware, which make it hard to rule-out / easy to be blind-sided by hardware and there requisite device changes. Regards, -- .\\ark ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Moving an OI install to a new hard drive?
Indeed. On VMware, you should install the "VMware Tools" for Solaris after installing the operating system. You will get better performance using the drivers written by the people who built the virtual hardware. As for the VMware Converter, I used it on Windows once; and, I believe there's a version for Linux, too. But, instead, I tarred-up my Linux systems, created a new VM, and with the assistance of a second "helper VM" (or rescue CD) I received and extracted the tar files; modified the kernel module load file for virtual drivers (based-on the configuration of an already running similar Linux VM); and, re-installed the boot loader. You could probably try the same with OpenIndiana; but, by the time you've made a "helper VM", you would probably be better-off just to copy files over to the new systems. :) Regards, -- .\\ark ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Gnome to become a Linux only project
If there's serious interest in getting KDE on OpenIndiana, maybe consider reaching-out to the Korona project. They essentially did live-DVD's, that piggy-backed the last public and development OpenSolaris releases. http://techbase.kde.org/Projects/KDE_on_Solaris/OpenSolaris/Korona However, myself originally being a KDE fan, I have recently become comfortably familiar enough with Gnome, to not worry about GUI wars anymore. Regards, -- .\\ark ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] oracle gives openoffuce to apache
TDP does make the source code available. It should just be a matter of determining dependencies, if any, and compiling. After that, how much work does it take to make an IPS package? :) -- .\\ark ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] Illumos / OpenIndiana podcast
On Mon, Jun 20, 2011 at 9:38 PM, Benediktus Anindito wrote: > i didn't understand German :| > The interviews are actually in English. Look at the times notated on the webcast, and listen within that vicinity. 3:29 Garrett D'Amore 17:16 Alasdair Lumsden Regards , -- .\\ark ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] OI-151: test drive
On Wed, Jun 22, 2011 at 3:11 AM, Alex Lam S.L. wrote: > <...> > Ctrl+Alt+F1 doesn't seem to help other than making the whole screen go > blank. > <...> This ain't Linux. So, that won't do anything unless you also build the virtual console terminals, which don't exist in any OpenSolaris derivative, by default. See: http://blogs.oracle.com/DanX/entry/solaris_virtual_consoles I'm not sure about your desktop freezing; but, being able to switch to command line should help you investigate / workaround the issue. Regards, -- .\\ark ___ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss
