Good day, I set up RAIDFRAME on HP DL380 G4 servers sometime in 2004 because we had problems with hardware RAID (SmartArray 6402 controller with 15000 rpm drives). We are no longer using RAIDFrame because OpenBSD 3.9 support for RAID on our machines is excellent and we no longer have problems with 15K rpm drives. So unfortunately, I am not able to get you a dmesg from this specific server when it was running RAIDFRAME.
Anyways, here is a cut and paste of what may be useful from my write up at that time. It was written for operators whose primary expertise is not OpenBSD, and the objective was just to help us rebuild our OpenBSD servers quickly if there were any hardware failures. So it just shows the steps to take within our environment and may not be appropriate for your situation. Sorry that it looks a bit disjointed but I had to delete all material that was specific to the data center. Hopefully it is still readable. If you want the OpenOffice version of this document, please let me know as well. Hope this helps, Vijay FROM THE SERVER CONFIGURATION DOCUMENT BASICS Install OpenBSD, using 4GB for /, 2GB for swap, and leave the rest for RAID Install KDE Base Make sure base installation is working properly. I modified the generic kernel with the following changes -- RAID support, a number of kernel optimizations including DUMMY_NOOPS, removal of support for 386/486/586 processors, no math emulation, and support for RAID Auto-configuration. I then built two custom kernels, one for single processors (bsd.vijay.sp) and one for SMP machines (bsd.vijay.mp). The objective was to provide similar or better performance than the Solaris boxes. Copy the two custom kernels built (bsd.vijay.mp and bsd.vijay.sp) to / Rename /bsd as /bsd.original and rename bsd.vijay.mp as /bsd I had copied the kernels to /home/vijay. So in this situation, I typed in the following commands: honyp34# cd / honyp34# mv bsd bsd.original honyp34# cp /home/vijay/bsd.vijay.mp /bsd This will replace the generic kernel with a custom kernel that supports RAID and is optimized for the Proliant DL380 G4 Server. Now comes the slightly complicated part. DISKLABEL disklabel sd0 > disklabel.sd0 You will get output as follows: # /dev/rsd0c: type: SCSI disk: SCSI disk label: BF07288576 flags: bytes/sector: 512 sectors/track: 723 tracks/cylinder: 4 sectors/cylinder: 2892 cylinders: 49158 total sectors: 142264000 rpm: 15000 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # microseconds track-to-track seek: 0 # microseconds drivedata: 0 16 partitions: # size offset fstype [fsize bsize cpg] a: 8389629 63 4.2BSD 2048 16384 416 # Cyl 0*- 2900 b: 4193400 8389692 swap # Cyl 2901 - 4350 c: 142264000 0 unused 0 0 # Cyl 0 - 49192* d: 129672483 12583092 RAID # Cyl 4351 - 49189* Now copy this to disklabel.sd1 and edit the new file so that we have: # /dev/rsd1c: type: SCSI disk: SCSI disk label: BF07288576 flags: bytes/sector: 512 sectors/track: 723 tracks/cylinder: 4 sectors/cylinder: 2892 cylinders: 49158 total sectors: 142264000 rpm: 15000 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # microseconds track-to-track seek: 0 # microseconds drivedata: 0 16 partitions: # size offset fstype [fsize bsize cpg] a: 8389629 63 4.2BSD 2048 16384 416 # Cyl 0*- 2900 b: 4193400 8389692 swap # Cyl 2901 - 4350 c: 142264000 0 unused 0 0 # Cyl 0 - 49192* d: 129672483 12583092 RAID # Cyl 4351 - 49189* MATCH DRIVES fdisk -i sd1 ----------------------------------------------------- ------ ATTENTION - UPDATING MASTER BOOT RECORD ------ ----------------------------------------------------- Do you wish to write new MBR and partition table? [n] y disklabel -R -r sd1 disklabel.sd1 # using MBR partition 3: type A6 off 63 (0x3f) size 142255512 (0x87aa598) Now, set up the new filesystem using the commands: newfs /dev/sd1a newfs /dev/sd1d Create and save a file called raid0.conf in /etc with the following contents: START array 1 2 0 START disks /dev/sd0d /dev/sd1d START layout 128 1 1 1 START queue fifo 100 mount /dev/sd1a /sd1 cd / pax -r -w -p e -v .profile bin boot bsd bsd.original dev dvd etc home mnt root sbin sd0 sys tmp usr var stand altroot /sd1/ dump -0f - / | restore -rf b is another alternative and is recommended by many people. I had problems with this command, when implementing a graphical OpenBSD. So resorted to the less popular pax. After the restore is completed successfully, copy the boot sectors to the second hard drive: /usr/mdec/installboot -v /sd1/boot /usr/mdec/biosboot sd1 Do not forget to edit fstab on the second hard drive so that /sd1/etc/fstab looks like: /dev/sd1a / ffs rw 1 1 Reboot so that we use the custom, RAID-enabled kernel. SET UP RAID If you followed all the steps listed above breligiouslyb then most probably you wonbt get too many errors. Most probably if you are running your commands from the console you may notice the following errors. Since raidctl ignores them, you can safely ignore them too! honyp34# raidctl -C /etc/raid0.conf raid0 Jun 21 10:19:33 honyp34 /bsd: vnode was NULL Jun 21 10:20:19 honyp34 /bsd: Hosed component: /dev/sd0d. Jun 21 10:20:19 honyp34 /bsd: Hosed component: /dev/sd0d. Jun 21 10:20:19 honyp34 /bsd: raid0: Ignoring /dev/sd0d. Jun 21 10:20:19 honyp34 /bsd: raid0: Ignoring /dev/sd0d. honyp34# Jun 21 10:20:19 honyp34 /bsd: raid0: Component /dev/sd1d being configured at row: 0 col: 1 Jun 21 10:20:19 honyp34 /bsd: raid0: Component /dev/sd1d being configured at row: 0 col: 1 Jun 21 10:20:19 honyp34 /bsd: Row: 16 Column: 24 Num Rows: 1200 Num Columns: 184 Jun 21 10:20:19 honyp34 /bsd: Row: 16 Column: 24 Num Rows: 1200 Num Columns: 184 Jun 21 10:20:19 honyp34 /bsd: Version: 0 Serial Number: 0 Mod Counter: 8 Jun 21 10:20:19 honyp34 /bsd: Version: 0 Serial Number: 0 Mod Counter: 8 Jun 21 10:20:19 honyp34 /bsd: Clean: Yes Status: 1119363391 Jun 21 10:20:19 honyp34 /bsd: Clean: Yes Status: 1119363391 Jun 21 10:20:19 honyp34 /bsd: Row out of alignment for: /dev/sd1d. Jun 21 10:20:19 honyp34 /bsd: Row out of alignment for: /dev/sd1d. Jun 21 10:20:19 honyp34 /bsd: Column out of alignment for: /dev/sd1d. Jun 21 10:20:19 honyp34 /bsd: Column out of alignment for: /dev/sd1d. Jun 21 10:20:19 honyp34 /bsd: Number of rows do not match for: /dev/sd1d. Jun 21 10:20:19 honyp34 /bsd: Number of rows do not match for: /dev/sd1d. Jun 21 10:20:19 honyp34 /bsd: Number of columns do not match for: /dev/sd1d. Jun 21 10:20:19 honyp34 /bsd: Number of columns do not match for: /dev/sd1d. Jun 21 10:20:19 honyp34 /bsd: raid0: There were fatal errors Jun 21 10:20:19 honyp34 /bsd: raid0: There were fatal errors Jun 21 10:20:19 honyp34 /bsd: raid0: Fatal errors being ignored. Jun 21 10:20:19 honyp34 /bsd: raid0: Fatal errors being ignored. Jun 21 10:20:19 honyp34 /bsd: raid0 (root) Jun 21 10:20:19 honyp34 /bsd: raid0 (root) honyp34# raidctl -I 100 raid0 Jun 21 10:20:38 honyp34 /bsd: raid0: no disk label Jun 21 10:20:38 honyp34 /bsd: raid0: no disk label honyp34# raidctl -iv raid0 Parity Re-Write status: 3% |* | ETA: 13:36 | honyp34# disklabel -E raid0 Jun 21 10:37:44 honyp34 /bsd: raid0: no disk label Jun 21 10:37:44 honyp34 /bsd: raid0: no disk label disklabel: Can't get bios geometry: Device not configured Initial label editor (enter '?' for help at any prompt) > a a offset: [0] size: [129672320] 4g FS type: [4.2BSD] > a b offset: [8388608] size: [121283712] 2g FS type: [swap] > a d offset: [12582912] size: [117089408] 4g FS type: [4.2BSD] > a e offset: [20971520] size: [108700800] 8g FS type: [4.2BSD] > a f offset: [37748736] size: [91923584] 8g FS type: [4.2BSD] > a g offset: [54525952] size: [75146368] FS type: [4.2BSD] > w > q No label changes. The following commands will set up new file systems on the RAID array. Please note that rraid stands for raw raid, a convention adopted from standard UNIX practice. For example, rsd0a stands for raw sd0a. newfs /dev/rraid0a newfs /dev/rraid0d newfs /dev/rraid0e newfs /dev/rraid0f newfs /dev/rraid0g I have included the five newfs commands above in the file construct-filesystems-1.sh so that you donbt have to type it in. This is the first step (as indicated by the -1 in the scriptbs name). Next, enter the following commands: mount /dev/raid0a /mnt cd /mnt mkdir tmp mkdir var mkdir usr mkdir home mount /dev/raid0d /mnt/tmp mount /dev/raid0e /mnt/var mount /dev/raid0f /mnt/usr mount /dev/raid0g /mnt/home I entered these commands in the file mount-raid-2.sh and you can alternately run that file to mount the partitions. This is step 2, indicated by the -2 in the file name. Then cd / pax -r -w -p e -v .profile bin boot bsd bsd.original dev dvd etc home root sbin sd0 sys tmp usr var stand altroot /mnt/ will copy OpenBSD installation. Carefully edit the file system table in /mnt/etc/fstab so that it looks like the following: /dev/raid0a / ffs rw 1 1 /dev/raid0b none swap sw 0 0 /dev/raid0d /tmp ffs rw,nodev,nosuid,softdep 1 2 /dev/raid0e /var ffs rw,nodev,nosuid,softdep 1 2 /dev/raid0f /usr ffs rw,nodev,softdep 1 2 /dev/raid0g /home ffs rw,nodev,nosuid,softdep 1 2 You may want to copy the file /home/vijay/script/fstab using the following command: honyp34# cp fstab /mnt/etc/fstab Now, we should make the RAID partitions bootable using the command: honyp34# raidctl -A root raid0 raid0: Autoconfigure: Yes raid0: Root: Yes CHECKS Checks Before proceeding further, do the following checks. You should not have any problems with each of the following reboots. If you do, first figure out what is wrong, fix it, and then do the checks again: 1. Reboot 2. Remove the first hard drive and reboot 3. Remove the second hard drive and reboot 4. Boot from hd1a:/bsd 5. Boot from hd0a/bsd With each of the checks: 1. You should be able to logon on as root 2. You should be able to start X-Windows 3. You should be able to start KDE On Wed, 2006-15-11 at 08:59 +0530, Siju George wrote: > Hi, > > I have been trying to configure software RAID using resources at > > http://www.openbsd.org/faq/faq14.html#RAID > > and > > http://www.openbsd.org/faq/faq14.html#RAID > > I did get the error > > # raidctl -C /root/raid0.conf raid0 > raidctl: ioctl (RAIDFRAME_CONFIGURE) failed > > The raid configuration file entries are > > ======================================================= > # cat /root/raid0.conf > START array > # numRow numCol numSpare > 1 2 0 > > START disks > /dev/wd2b # the fake device > /dev/wd1b > > START layout > 128 1 1 1 > > START queue > fifo 100 > ================================================ > > Could some one help me trouble shoot this please? > I did recompile the kernel with the following options and am booting that > kernel > ================================================= > # cat /usr/src/sys/arch/amd64/conf/GENERIC.RAID > include "arch/amd64/conf/GENERIC" # include GENERIC configuration > option RAID_AUTOCONFIG # automatically configure RAIDframe arrays on boot > pseudo-device raid 4 # RAIDframe disk driver > # uname -a > OpenBSD xxx.yyy.local 4.0 GENERIC.RAID#0 amd64 > # > ========================================================== > > # disklabel -E wd1 > # Inside MBR partition 3: type A6 start 63 size 234436482 > > Treating sectors 63-234436545 as the OpenBSD portion of the disk. > You can use the 'b' command to change this. > > Initial label editor (enter '?' for help at any prompt) > > p m > device: /dev/rwd1c > type: ESDI > disk: ESDI/IDE disk > label: ST3120827AS > bytes/sector: 512 > sectors/track: 63 > tracks/cylinder: 16 > sectors/cylinder: 1008 > cylinders: 16383 > total bytes: 114473.5M > free bytes: 114470.9M > rpm: 3600 > > 16 partitions: > # size offset fstype [fsize bsize cpg] > c: 114473.5M 0.0M unused 0 0 # Cyl 0 -232580 > > a a > offset: [63] > size: [234436482] 1024m > Rounding to nearest cylinder: 2097585 > FS type: [4.2BSD] > > a b > offset: [2097648] > size: [232338897] RAID > Invalid entry > size: [232338897] > FS type: [swap] RAID > > p m > device: /dev/rwd1c > type: ESDI > disk: ESDI/IDE disk > label: ST3120827AS > bytes/sector: 512 > sectors/track: 63 > tracks/cylinder: 16 > sectors/cylinder: 1008 > cylinders: 16383 > total bytes: 114473.5M > free bytes: 0.0M > rpm: 3600 > > 16 partitions: > # size offset fstype [fsize bsize cpg] > a: 1024.2M 0.0M 4.2BSD 2048 16384 16 # Cyl 0*- 2080 > b: 113446.7M 1024.2M RAID # Cyl 2081 > -232575* > c: 114473.5M 0.0M unused 0 0 # Cyl 0 -232580 > > q > Write new label?: [y] > # newfs wd1a > Warning: 64 sector(s) in last cylinder unallocated > /dev/rwd1a: 2097584 sectors in 2081 cylinders of 16 tracks, 63 sectors > 1024.2MB in 7 cyl groups (328 c/g, 161.44MB/g, 20608 i/g) > super-block backups (for fsck -b #) at: > 32, 330720, 661408, 992096, 1322784, 1653472, 1984160, > # mount /dev/wd1a /mnt > # cp /bsd /usr/mdec/boot /mnt > # /usr/mdec/installboot -v /mnt/boot /usr/mdec/biosboot wd1 > boot: /mnt/boot > proto: /usr/mdec/biosboot > device: /dev/rwd1c > /usr/mdec/biosboot: entry point 0 > proto bootblock size 512 > /mnt/boot is 3 blocks x 16384 bytes > fs block shift 2; part offset 63; inode block 24, offset 936 > using MBR partition 3: type 166 (0xa6) offset 63 (0x3f) > # cat >> /root/raid0.conf << EOF > > START array > > # numRow numCol numSpare > > 1 2 0 > > > > START disks > > /dev/wd2b # the fake device > > /dev/wd1b > > > > START layout > > 128 1 1 1 > > > > START queue > > fifo 100 > > EOF > # raidctl -C /root/raid0.conf raid0 > raidctl: ioctl (RAIDFRAME_CONFIGURE) failed > -- Vijay Sankar, M.Eng., P.Eng. ForeTell Technologies Limited 59 Flamingo Avenue, Winnipeg, MB, Canada R3J 0X6 Phone: 204 885 9535, E-Mail: [EMAIL PROTECTED]