Good day,
I set up RAIDFRAME on HP DL380 G4 servers sometime in 2004 because we
had problems with hardware RAID (SmartArray 6402 controller with 15000
rpm drives). We are no longer using RAIDFrame because OpenBSD 3.9
support for RAID on our machines is excellent and we no longer have
problems with 15K rpm drives. So unfortunately, I am not able to get you
a dmesg from this specific server when it was running RAIDFRAME.
Anyways, here is a cut and paste of what may be useful from my write up
at that time. It was written for operators whose primary expertise is
not OpenBSD, and the objective was just to help us rebuild our OpenBSD
servers quickly if there were any hardware failures. So it just shows
the steps to take within our environment and may not be appropriate for
your situation. Sorry that it looks a bit disjointed but I had to delete
all material that was specific to the data center. Hopefully it is still
readable. If you want the OpenOffice version of this document, please
let me know as well.
Hope this helps,
Vijay
FROM THE SERVER CONFIGURATION DOCUMENT
BASICS
Install OpenBSD, using 4GB for /, 2GB for swap, and leave the rest for
RAID
Install KDE Base
Make sure base installation is working properly.
I modified the generic kernel with the following changes -- RAID
support, a number of kernel optimizations including DUMMY_NOOPS, removal
of support for 386/486/586 processors, no math emulation, and support
for RAID Auto-configuration. I then built two custom kernels, one for
single processors (bsd.vijay.sp) and one for SMP machines
(bsd.vijay.mp). The objective was to provide similar or better
performance than the Solaris boxes. Copy the two custom kernels built
(bsd.vijay.mp and bsd.vijay.sp) to /
Rename /bsd as /bsd.original and rename bsd.vijay.mp as /bsd
I had copied the kernels to /home/vijay. So in this situation, I typed
in the following commands:
honyp34# cd /
honyp34# mv bsd bsd.original
honyp34# cp /home/vijay/bsd.vijay.mp /bsd
This will replace the generic kernel with a custom kernel that supports
RAID and is optimized for the Proliant DL380 G4 Server.
Now comes the slightly complicated part.
DISKLABEL
disklabel sd0 > disklabel.sd0
You will get output as follows:
# /dev/rsd0c:
type: SCSI
disk: SCSI disk
label: BF07288576
flags:
bytes/sector: 512
sectors/track: 723
tracks/cylinder: 4
sectors/cylinder: 2892
cylinders: 49158
total sectors: 142264000
rpm: 15000
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 8389629 63 4.2BSD 2048 16384 416 # Cyl 0*- 2900
b: 4193400 8389692 swap # Cyl 2901 - 4350
c: 142264000 0 unused 0 0 # Cyl 0 - 49192*
d: 129672483 12583092 RAID # Cyl 4351 - 49189*
Now copy this to disklabel.sd1 and edit the new file so that we have:
# /dev/rsd1c:
type: SCSI
disk: SCSI disk
label: BF07288576
flags:
bytes/sector: 512
sectors/track: 723
tracks/cylinder: 4
sectors/cylinder: 2892
cylinders: 49158
total sectors: 142264000
rpm: 15000
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 8389629 63 4.2BSD 2048 16384 416 # Cyl 0*- 2900
b: 4193400 8389692 swap # Cyl 2901 - 4350
c: 142264000 0 unused 0 0 # Cyl 0 - 49192*
d: 129672483 12583092 RAID # Cyl 4351 - 49189*
MATCH DRIVES
fdisk -i sd1
-----------------------------------------------------
------ ATTENTION - UPDATING MASTER BOOT RECORD ------
-----------------------------------------------------
Do you wish to write new MBR and partition table? [n] y
disklabel -R -r sd1 disklabel.sd1
# using MBR partition 3: type A6 off 63 (0x3f) size 142255512
(0x87aa598)
Now, set up the new filesystem using the commands:
newfs /dev/sd1a
newfs /dev/sd1d
Create and save a file called raid0.conf in /etc with the following
contents:
START array
1 2 0
START disks
/dev/sd0d
/dev/sd1d
START layout
128 1 1 1
START queue
fifo 100
mount /dev/sd1a /sd1
cd /
pax -r -w -p e -v .profile bin boot bsd bsd.original dev dvd etc home
mnt root sbin sd0 sys tmp usr var stand altroot /sd1/
dump -0f - / | restore -rf b is another alternative and is recommended
by many people. I had problems with this command, when implementing a
graphical OpenBSD. So resorted to the less popular pax.
After the restore is completed successfully, copy the boot sectors to
the second hard drive:
/usr/mdec/installboot -v /sd1/boot /usr/mdec/biosboot sd1
Do not forget to edit fstab on the second hard drive so
that /sd1/etc/fstab looks like:
/dev/sd1a / ffs rw 1 1
Reboot so that we use the custom, RAID-enabled kernel.
SET UP RAID
If you followed all the steps listed above b religiouslyb then most
probably you wonbt get too many errors. Most probably if you are running
your commands from the console you may notice the following errors.
Since raidctl ignores them, you can safely ignore them too!
honyp34# raidctl -C /etc/raid0.conf raid0
Jun 21 10:19:33 honyp34 /bsd: vnode was NULL
Jun 21 10:20:19 honyp34 /bsd: Hosed component: /dev/sd0d.
Jun 21 10:20:19 honyp34 /bsd: Hosed component: /dev/sd0d.
Jun 21 10:20:19 honyp34 /bsd: raid0: Ignoring /dev/sd0d.
Jun 21 10:20:19 honyp34 /bsd: raid0: Ignoring /dev/sd0d.
honyp34# Jun 21 10:20:19 honyp34 /bsd: raid0: Component /dev/sd1d being
configured at row: 0 col: 1
Jun 21 10:20:19 honyp34 /bsd: raid0: Component /dev/sd1d being
configured at row: 0 col: 1
Jun 21 10:20:19 honyp34 /bsd: Row: 16 Column: 24 Num Rows: 1200 Num
Columns: 184
Jun 21 10:20:19 honyp34 /bsd: Row: 16 Column: 24 Num Rows: 1200 Num
Columns: 184
Jun 21 10:20:19 honyp34 /bsd: Version: 0 Serial Number: 0 Mod Counter: 8
Jun 21 10:20:19 honyp34 /bsd: Version: 0 Serial Number: 0 Mod Counter: 8
Jun 21 10:20:19 honyp34 /bsd: Clean: Yes Status: 1119363391
Jun 21 10:20:19 honyp34 /bsd: Clean: Yes Status: 1119363391
Jun 21 10:20:19 honyp34 /bsd: Row out of alignment for: /dev/sd1d.
Jun 21 10:20:19 honyp34 /bsd: Row out of alignment for: /dev/sd1d.
Jun 21 10:20:19 honyp34 /bsd: Column out of alignment for: /dev/sd1d.
Jun 21 10:20:19 honyp34 /bsd: Column out of alignment for: /dev/sd1d.
Jun 21 10:20:19 honyp34 /bsd: Number of rows do not match
for: /dev/sd1d.
Jun 21 10:20:19 honyp34 /bsd: Number of rows do not match
for: /dev/sd1d.
Jun 21 10:20:19 honyp34 /bsd: Number of columns do not match
for: /dev/sd1d.
Jun 21 10:20:19 honyp34 /bsd: Number of columns do not match
for: /dev/sd1d.
Jun 21 10:20:19 honyp34 /bsd: raid0: There were fatal errors
Jun 21 10:20:19 honyp34 /bsd: raid0: There were fatal errors
Jun 21 10:20:19 honyp34 /bsd: raid0: Fatal errors being ignored.
Jun 21 10:20:19 honyp34 /bsd: raid0: Fatal errors being ignored.
Jun 21 10:20:19 honyp34 /bsd: raid0 (root)
Jun 21 10:20:19 honyp34 /bsd: raid0 (root)
honyp34# raidctl -I 100 raid0
Jun 21 10:20:38 honyp34 /bsd: raid0: no disk label
Jun 21 10:20:38 honyp34 /bsd: raid0: no disk label
honyp34# raidctl -iv raid0
Parity Re-Write status:
3% |* | ETA: 13:36 |
honyp34# disklabel -E raid0
Jun 21 10:37:44 honyp34 /bsd: raid0: no disk label
Jun 21 10:37:44 honyp34 /bsd: raid0: no disk label
disklabel: Can't get bios geometry: Device not configured
Initial label editor (enter '?' for help at any prompt)
> a a
offset: [0]
size: [129672320] 4g
FS type: [4.2BSD]
> a b
offset: [8388608]
size: [121283712] 2g
FS type: [swap]
> a d
offset: [12582912]
size: [117089408] 4g
FS type: [4.2BSD]
> a e
offset: [20971520]
size: [108700800] 8g
FS type: [4.2BSD]
> a f
offset: [37748736]
size: [91923584] 8g
FS type: [4.2BSD]
> a g
offset: [54525952]
size: [75146368]
FS type: [4.2BSD]
> w
> q
No label changes.
The following commands will set up new file systems on the RAID array.
Please note that rraid stands for raw raid, a convention adopted from
standard UNIX practice. For example, rsd0a stands for raw sd0a.
newfs /dev/rraid0a
newfs /dev/rraid0d
newfs /dev/rraid0e
newfs /dev/rraid0f
newfs /dev/rraid0g
I have included the five newfs commands above in the file
construct-filesystems-1.sh so that you donbt have to type it in. This is
the first step (as indicated by the -1 in the scriptbs name).
Next, enter the following commands:
mount /dev/raid0a /mnt
cd /mnt
mkdir tmp
mkdir var
mkdir usr
mkdir home
mount /dev/raid0d /mnt/tmp
mount /dev/raid0e /mnt/var
mount /dev/raid0f /mnt/usr
mount /dev/raid0g /mnt/home
I entered these commands in the file mount-raid-2.sh and you can
alternately run that file to mount the partitions. This is step 2,
indicated by the -2 in the file name. Then
cd /
pax -r -w -p e -v .profile bin boot bsd bsd.original dev dvd etc home
root sbin sd0 sys tmp usr var stand altroot /mnt/
will copy OpenBSD installation.
Carefully edit the file system table in /mnt/etc/fstab so that it looks
like the following:
/dev/raid0a / ffs rw 1 1
/dev/raid0b none swap sw 0 0
/dev/raid0d /tmp ffs rw,nodev,nosuid,softdep 1 2
/dev/raid0e /var ffs rw,nodev,nosuid,softdep 1 2
/dev/raid0f /usr ffs rw,nodev,softdep 1 2
/dev/raid0g /home ffs rw,nodev,nosuid,softdep 1 2
You may want to copy the file /home/vijay/script/fstab using the
following command:
honyp34# cp fstab /mnt/etc/fstab
Now, we should make the RAID partitions bootable using the command:
honyp34# raidctl -A root raid0
raid0: Autoconfigure: Yes
raid0: Root: Yes
CHECKS
Checks
Before proceeding further, do the following checks. You should not have
any problems with each of the following reboots. If you do, first figure
out what is wrong, fix it, and then do the checks again:
1. Reboot
2. Remove the first hard drive and reboot
3. Remove the second hard drive and reboot
4. Boot from hd1a:/bsd
5. Boot from hd0a/bsd
With each of the checks:
1. You should be able to logon on as root
2. You should be able to start X-Windows
3. You should be able to start KDE
On Wed, 2006-15-11 at 08:59 +0530, Siju George wrote:
> Hi,
>
> I have been trying to configure software RAID using resources at
>
> http://www.openbsd.org/faq/faq14.html#RAID
>
> and
>
> http://www.openbsd.org/faq/faq14.html#RAID
>
> I did get the error
>
> # raidctl -C /root/raid0.conf raid0
> raidctl: ioctl (RAIDFRAME_CONFIGURE) failed
>
> The raid configuration file entries are
>
> =======================================================
> # cat /root/raid0.conf
> START array
> # numRow numCol numSpare
> 1 2 0
>
> START disks
> /dev/wd2b # the fake device
> /dev/wd1b
>
> START layout
> 128 1 1 1
>
> START queue
> fifo 100
> ================================================
>
> Could some one help me trouble shoot this please?
> I did recompile the kernel with the following options and am booting that
> kernel
> =================================================
> # cat /usr/src/sys/arch/amd64/conf/GENERIC.RAID
> include "arch/amd64/conf/GENERIC" # include GENERIC configuration
> option RAID_AUTOCONFIG # automatically configure RAIDframe arrays on boot
> pseudo-device raid 4 # RAIDframe disk driver
> # uname -a
> OpenBSD xxx.yyy.local 4.0 GENERIC.RAID#0 amd64
> #
> ==========================================================
>
> # disklabel -E wd1
> # Inside MBR partition 3: type A6 start 63 size 234436482
>
> Treating sectors 63-234436545 as the OpenBSD portion of the disk.
> You can use the 'b' command to change this.
>
> Initial label editor (enter '?' for help at any prompt)
> > p m
> device: /dev/rwd1c
> type: ESDI
> disk: ESDI/IDE disk
> label: ST3120827AS
> bytes/sector: 512
> sectors/track: 63
> tracks/cylinder: 16
> sectors/cylinder: 1008
> cylinders: 16383
> total bytes: 114473.5M
> free bytes: 114470.9M
> rpm: 3600
>
> 16 partitions:
> # size offset fstype [fsize bsize cpg]
> c: 114473.5M 0.0M unused 0 0 # Cyl 0 -232580
> > a a
> offset: [63]
> size: [234436482] 1024m
> Rounding to nearest cylinder: 2097585
> FS type: [4.2BSD]
> > a b
> offset: [2097648]
> size: [232338897] RAID
> Invalid entry
> size: [232338897]
> FS type: [swap] RAID
> > p m
> device: /dev/rwd1c
> type: ESDI
> disk: ESDI/IDE disk
> label: ST3120827AS
> bytes/sector: 512
> sectors/track: 63
> tracks/cylinder: 16
> sectors/cylinder: 1008
> cylinders: 16383
> total bytes: 114473.5M
> free bytes: 0.0M
> rpm: 3600
>
> 16 partitions:
> # size offset fstype [fsize bsize cpg]
> a: 1024.2M 0.0M 4.2BSD 2048 16384 16 # Cyl 0*- 2080
> b: 113446.7M 1024.2M RAID # Cyl 2081
> -232575*
> c: 114473.5M 0.0M unused 0 0 # Cyl 0 -232580
> > q
> Write new label?: [y]
> # newfs wd1a
> Warning: 64 sector(s) in last cylinder unallocated
> /dev/rwd1a: 2097584 sectors in 2081 cylinders of 16 tracks, 63 sectors
> 1024.2MB in 7 cyl groups (328 c/g, 161.44MB/g, 20608 i/g)
> super-block backups (for fsck -b #) at:
> 32, 330720, 661408, 992096, 1322784, 1653472, 1984160,
> # mount /dev/wd1a /mnt
> # cp /bsd /usr/mdec/boot /mnt
> # /usr/mdec/installboot -v /mnt/boot /usr/mdec/biosboot wd1
> boot: /mnt/boot
> proto: /usr/mdec/biosboot
> device: /dev/rwd1c
> /usr/mdec/biosboot: entry point 0
> proto bootblock size 512
> /mnt/boot is 3 blocks x 16384 bytes
> fs block shift 2; part offset 63; inode block 24, offset 936
> using MBR partition 3: type 166 (0xa6) offset 63 (0x3f)
> # cat >> /root/raid0.conf << EOF
> > START array
> > # numRow numCol numSpare
> > 1 2 0
> >
> > START disks
> > /dev/wd2b # the fake device
> > /dev/wd1b
> >
> > START layout
> > 128 1 1 1
> >
> > START queue
> > fifo 100
> > EOF
> # raidctl -C /root/raid0.conf raid0
> raidctl: ioctl (RAIDFRAME_CONFIGURE) failed
>
--
Vijay Sankar, M.Eng., P.Eng.
ForeTell Technologies Limited
59 Flamingo Avenue, Winnipeg, MB, Canada R3J 0X6
Phone: 204 885 9535, E-Mail: [EMAIL PROTECTED]
