Good day,

I set up RAIDFRAME on HP DL380 G4 servers sometime in 2004 because we
had problems with hardware RAID (SmartArray 6402 controller with 15000
rpm drives). We are no longer using RAIDFrame because OpenBSD 3.9
support for RAID on our machines is excellent and we no longer have
problems with 15K rpm drives. So unfortunately, I am not able to get you
a dmesg from this specific server when it was running RAIDFRAME.

Anyways, here is a cut and paste of what may be useful from my write up
at that time. It was written for operators whose primary expertise is
not OpenBSD, and the objective was just to help us rebuild our OpenBSD
servers quickly if there were any hardware failures. So it just shows
the steps to take within our environment and may not be appropriate for
your situation. Sorry that it looks a bit disjointed but I had to delete
all material that was specific to the data center. Hopefully it is still
readable. If you want the OpenOffice version of this document, please
let me know as well. 

Hope this helps,

Vijay

FROM THE SERVER CONFIGURATION DOCUMENT

BASICS

Install OpenBSD, using 4GB for /, 2GB for swap, and leave the rest for
RAID

Install KDE Base

Make sure base installation is working properly.

I modified the generic kernel with the following changes -- RAID
support, a number of kernel optimizations including DUMMY_NOOPS, removal
of support for 386/486/586 processors, no math emulation, and support
for RAID Auto-configuration. I then built two custom kernels, one for
single processors (bsd.vijay.sp) and one for SMP machines
(bsd.vijay.mp). The objective was to provide similar or better
performance than the Solaris boxes. Copy the two custom kernels built
(bsd.vijay.mp and bsd.vijay.sp) to /


Rename /bsd as /bsd.original and rename bsd.vijay.mp as /bsd


I had copied the kernels to /home/vijay. So in this situation, I typed
in the following commands:


honyp34# cd /

honyp34# mv bsd bsd.original

honyp34# cp /home/vijay/bsd.vijay.mp /bsd


This will replace the generic kernel with a custom kernel that supports
RAID and is optimized for the Proliant DL380 G4 Server.


Now comes the slightly complicated part.


DISKLABEL
disklabel sd0 > disklabel.sd0


You will get output as follows:


# /dev/rsd0c:

type: SCSI

disk: SCSI disk

label: BF07288576

flags:

bytes/sector: 512

sectors/track: 723

tracks/cylinder: 4

sectors/cylinder: 2892

cylinders: 49158

total sectors: 142264000

rpm: 15000

interleave: 1

trackskew: 0

cylinderskew: 0

headswitch: 0 # microseconds

track-to-track seek: 0 # microseconds

drivedata: 0


16 partitions:

# size offset fstype [fsize bsize cpg]

a: 8389629 63 4.2BSD 2048 16384 416 # Cyl 0*- 2900

b: 4193400 8389692 swap # Cyl 2901 - 4350

c: 142264000 0 unused 0 0 # Cyl 0 - 49192*

d: 129672483 12583092 RAID # Cyl 4351 - 49189*


Now copy this to disklabel.sd1 and edit the new file so that we have:


# /dev/rsd1c:

type: SCSI

disk: SCSI disk

label: BF07288576

flags:

bytes/sector: 512

sectors/track: 723

tracks/cylinder: 4

sectors/cylinder: 2892

cylinders: 49158

total sectors: 142264000

rpm: 15000

interleave: 1

trackskew: 0

cylinderskew: 0

headswitch: 0 # microseconds

track-to-track seek: 0 # microseconds

drivedata: 0


16 partitions:

# size offset fstype [fsize bsize cpg]

a: 8389629 63 4.2BSD 2048 16384 416 # Cyl 0*- 2900

b: 4193400 8389692 swap # Cyl 2901 - 4350

c: 142264000 0 unused 0 0 # Cyl 0 - 49192*

d: 129672483 12583092 RAID # Cyl 4351 - 49189*


MATCH DRIVES

fdisk -i sd1


-----------------------------------------------------

------ ATTENTION - UPDATING MASTER BOOT RECORD ------

-----------------------------------------------------


Do you wish to write new MBR and partition table? [n] y


disklabel -R -r sd1 disklabel.sd1

# using MBR partition 3: type A6 off 63 (0x3f) size 142255512
(0x87aa598)


Now, set up the new filesystem using the commands:


newfs /dev/sd1a

newfs /dev/sd1d


Create and save a file called raid0.conf in /etc with the following
contents:


START array

1 2 0

START disks

/dev/sd0d

/dev/sd1d

START layout

128 1 1 1

START queue

fifo 100


mount /dev/sd1a /sd1


cd /


pax -r -w -p e -v .profile bin boot bsd bsd.original dev dvd etc home
mnt root sbin sd0 sys tmp usr var stand altroot /sd1/


dump -0f - / | restore -rf b is another alternative and is recommended
by many people. I had problems with this command, when implementing a
graphical OpenBSD. So resorted to the less popular pax.


After the restore is completed successfully, copy the boot sectors to
the second hard drive:


/usr/mdec/installboot -v /sd1/boot /usr/mdec/biosboot sd1


Do not forget to edit fstab on the second hard drive so
that /sd1/etc/fstab looks like:


/dev/sd1a / ffs rw 1 1


Reboot so that we use the custom, RAID-enabled kernel.


SET UP RAID

If you followed all the steps listed above breligiouslyb then most
probably you wonbt get too many errors. Most probably if you are running
your commands from the console you may notice the following errors.
Since raidctl ignores them, you can safely ignore them too!


honyp34# raidctl -C /etc/raid0.conf raid0

Jun 21 10:19:33 honyp34 /bsd: vnode was NULL

Jun 21 10:20:19 honyp34 /bsd: Hosed component: /dev/sd0d.

Jun 21 10:20:19 honyp34 /bsd: Hosed component: /dev/sd0d.

Jun 21 10:20:19 honyp34 /bsd: raid0: Ignoring /dev/sd0d.

Jun 21 10:20:19 honyp34 /bsd: raid0: Ignoring /dev/sd0d.

honyp34# Jun 21 10:20:19 honyp34 /bsd: raid0: Component /dev/sd1d being
configured at row: 0 col: 1

Jun 21 10:20:19 honyp34 /bsd: raid0: Component /dev/sd1d being
configured at row: 0 col: 1

Jun 21 10:20:19 honyp34 /bsd: Row: 16 Column: 24 Num Rows: 1200 Num
Columns: 184

Jun 21 10:20:19 honyp34 /bsd: Row: 16 Column: 24 Num Rows: 1200 Num
Columns: 184

Jun 21 10:20:19 honyp34 /bsd: Version: 0 Serial Number: 0 Mod Counter: 8

Jun 21 10:20:19 honyp34 /bsd: Version: 0 Serial Number: 0 Mod Counter: 8

Jun 21 10:20:19 honyp34 /bsd: Clean: Yes Status: 1119363391

Jun 21 10:20:19 honyp34 /bsd: Clean: Yes Status: 1119363391 

Jun 21 10:20:19 honyp34 /bsd: Row out of alignment for: /dev/sd1d.

Jun 21 10:20:19 honyp34 /bsd: Row out of alignment for: /dev/sd1d.

Jun 21 10:20:19 honyp34 /bsd: Column out of alignment for: /dev/sd1d.

Jun 21 10:20:19 honyp34 /bsd: Column out of alignment for: /dev/sd1d.

Jun 21 10:20:19 honyp34 /bsd: Number of rows do not match
for: /dev/sd1d.

Jun 21 10:20:19 honyp34 /bsd: Number of rows do not match
for: /dev/sd1d.

Jun 21 10:20:19 honyp34 /bsd: Number of columns do not match
for: /dev/sd1d.

Jun 21 10:20:19 honyp34 /bsd: Number of columns do not match
for: /dev/sd1d.

Jun 21 10:20:19 honyp34 /bsd: raid0: There were fatal errors

Jun 21 10:20:19 honyp34 /bsd: raid0: There were fatal errors

Jun 21 10:20:19 honyp34 /bsd: raid0: Fatal errors being ignored.

Jun 21 10:20:19 honyp34 /bsd: raid0: Fatal errors being ignored.

Jun 21 10:20:19 honyp34 /bsd: raid0 (root)

Jun 21 10:20:19 honyp34 /bsd: raid0 (root)


honyp34# raidctl -I 100 raid0

Jun 21 10:20:38 honyp34 /bsd: raid0: no disk label

Jun 21 10:20:38 honyp34 /bsd: raid0: no disk label


honyp34# raidctl -iv raid0

Parity Re-Write status:

3% |* | ETA: 13:36 |


honyp34# disklabel -E raid0

Jun 21 10:37:44 honyp34 /bsd: raid0: no disk label

Jun 21 10:37:44 honyp34 /bsd: raid0: no disk label

disklabel: Can't get bios geometry: Device not configured


Initial label editor (enter '?' for help at any prompt)

> a a

offset: [0]

size: [129672320] 4g

FS type: [4.2BSD]

> a b

offset: [8388608]

size: [121283712] 2g

FS type: [swap]

> a d

offset: [12582912]

size: [117089408] 4g

FS type: [4.2BSD]

> a e

offset: [20971520]

size: [108700800] 8g

FS type: [4.2BSD]

> a f

offset: [37748736]

size: [91923584] 8g

FS type: [4.2BSD]

> a g

offset: [54525952]

size: [75146368]

FS type: [4.2BSD]

> w

> q

No label changes.


The following commands will set up new file systems on the RAID array.
Please note that rraid stands for raw raid, a convention adopted from
standard UNIX practice. For example, rsd0a stands for raw sd0a.


newfs /dev/rraid0a

newfs /dev/rraid0d

newfs /dev/rraid0e

newfs /dev/rraid0f

newfs /dev/rraid0g


I have included the five newfs commands above in the file
construct-filesystems-1.sh so that you donbt have to type it in. This is
the first step (as indicated by the -1 in the scriptbs name).


Next, enter the following commands:


mount /dev/raid0a /mnt

cd /mnt

mkdir tmp

mkdir var

mkdir usr

mkdir home

mount /dev/raid0d /mnt/tmp

mount /dev/raid0e /mnt/var

mount /dev/raid0f /mnt/usr

mount /dev/raid0g /mnt/home


I entered these commands in the file mount-raid-2.sh and you can
alternately run that file to mount the partitions. This is step 2,
indicated by the -2 in the file name. Then


cd /

pax -r -w -p e -v .profile bin boot bsd bsd.original dev dvd etc home
root sbin sd0 sys tmp usr var stand altroot /mnt/


will copy OpenBSD installation. 


Carefully edit the file system table in /mnt/etc/fstab so that it looks
like the following:


/dev/raid0a / ffs rw 1 1

/dev/raid0b none swap sw 0 0

/dev/raid0d /tmp ffs rw,nodev,nosuid,softdep 1 2

/dev/raid0e /var ffs rw,nodev,nosuid,softdep 1 2

/dev/raid0f /usr ffs rw,nodev,softdep 1 2

/dev/raid0g /home ffs rw,nodev,nosuid,softdep 1 2


You may want to copy the file /home/vijay/script/fstab using the
following command:


honyp34# cp fstab /mnt/etc/fstab


Now, we should make the RAID partitions bootable using the command:


honyp34# raidctl -A root raid0


raid0: Autoconfigure: Yes

raid0: Root: Yes

CHECKS

Checks
Before proceeding further, do the following checks. You should not have
any problems with each of the following reboots. If you do, first figure
out what is wrong, fix it, and then do the checks again:


     1. Reboot
        
     2. Remove the first hard drive and reboot
        
     3. Remove the second hard drive and reboot
        
     4. Boot from hd1a:/bsd
        
     5. Boot from hd0a/bsd
        


With each of the checks:


             1. You should be able to logon on as root
                
             2. You should be able to start X-Windows
                
             3. You should be able to start KDE
                


On Wed, 2006-15-11 at 08:59 +0530, Siju George wrote:
> Hi,
> 
>  I have been trying to configure software RAID using resources at
> 
> http://www.openbsd.org/faq/faq14.html#RAID
> 
> and
> 
> http://www.openbsd.org/faq/faq14.html#RAID
> 
> I did get the error
> 
> # raidctl -C /root/raid0.conf raid0
> raidctl: ioctl (RAIDFRAME_CONFIGURE) failed
> 
> The raid configuration file entries are
> 
> =======================================================
> # cat /root/raid0.conf
> START array
> # numRow numCol numSpare
> 1 2 0
> 
> START disks
> /dev/wd2b # the fake device
> /dev/wd1b
> 
> START layout
> 128 1 1 1
> 
> START queue
> fifo 100
> ================================================
> 
> Could some one help me trouble shoot this please?
> I did recompile the kernel with the following options and am booting that 
> kernel
> =================================================
> # cat /usr/src/sys/arch/amd64/conf/GENERIC.RAID
> include "arch/amd64/conf/GENERIC" # include GENERIC configuration
> option RAID_AUTOCONFIG # automatically configure RAIDframe arrays on boot
> pseudo-device raid 4 # RAIDframe disk driver
> # uname -a
> OpenBSD xxx.yyy.local 4.0 GENERIC.RAID#0 amd64
> #
> ==========================================================
> 
> # disklabel -E wd1
> # Inside MBR partition 3: type A6 start 63 size 234436482
> 
> Treating sectors 63-234436545 as the OpenBSD portion of the disk.
> You can use the 'b' command to change this.
> 
> Initial label editor (enter '?' for help at any prompt)
> > p m
> device: /dev/rwd1c
> type: ESDI
> disk: ESDI/IDE disk
> label: ST3120827AS
> bytes/sector: 512
> sectors/track: 63
> tracks/cylinder: 16
> sectors/cylinder: 1008
> cylinders: 16383
> total bytes: 114473.5M
> free bytes: 114470.9M
> rpm: 3600
> 
> 16 partitions:
> #             size        offset  fstype [fsize bsize  cpg]
>   c:     114473.5M          0.0M  unused      0     0      # Cyl     0 -232580
> > a a
> offset: [63]
> size: [234436482] 1024m
> Rounding to nearest cylinder: 2097585
> FS type: [4.2BSD]
> > a b
> offset: [2097648]
> size: [232338897] RAID
> Invalid entry
> size: [232338897]
> FS type: [swap] RAID
> > p m
> device: /dev/rwd1c
> type: ESDI
> disk: ESDI/IDE disk
> label: ST3120827AS
> bytes/sector: 512
> sectors/track: 63
> tracks/cylinder: 16
> sectors/cylinder: 1008
> cylinders: 16383
> total bytes: 114473.5M
> free bytes: 0.0M
> rpm: 3600
> 
> 16 partitions:
> #             size        offset  fstype [fsize bsize  cpg]
>   a:       1024.2M          0.0M  4.2BSD   2048 16384   16 # Cyl     0*-  2080
>   b:     113446.7M       1024.2M    RAID                   # Cyl  2081 
> -232575*
>   c:     114473.5M          0.0M  unused      0     0      # Cyl     0 -232580
> > q
> Write new label?: [y]
> # newfs wd1a
> Warning: 64 sector(s) in last cylinder unallocated
> /dev/rwd1a:     2097584 sectors in 2081 cylinders of 16 tracks, 63 sectors
>         1024.2MB in 7 cyl groups (328 c/g, 161.44MB/g, 20608 i/g)
> super-block backups (for fsck -b #) at:
>  32, 330720, 661408, 992096, 1322784, 1653472, 1984160,
> # mount /dev/wd1a /mnt
> # cp /bsd /usr/mdec/boot /mnt
> # /usr/mdec/installboot -v /mnt/boot /usr/mdec/biosboot wd1
> boot: /mnt/boot
> proto: /usr/mdec/biosboot
> device: /dev/rwd1c
> /usr/mdec/biosboot: entry point 0
> proto bootblock size 512
> /mnt/boot is 3 blocks x 16384 bytes
> fs block shift 2; part offset 63; inode block 24, offset 936
> using MBR partition 3: type 166 (0xa6) offset 63 (0x3f)
> # cat >> /root/raid0.conf << EOF
> > START array
> > # numRow numCol numSpare
> > 1 2 0
> >
> > START disks
> > /dev/wd2b # the fake device
> > /dev/wd1b
> >
> > START layout
> > 128 1 1 1
> >
> > START queue
> > fifo 100
> > EOF
> # raidctl -C /root/raid0.conf raid0
> raidctl: ioctl (RAIDFRAME_CONFIGURE) failed
> 
-- 
Vijay Sankar, M.Eng., P.Eng.
ForeTell Technologies Limited
59 Flamingo Avenue, Winnipeg, MB, Canada R3J 0X6
Phone: 204 885 9535, E-Mail: [EMAIL PROTECTED]

Reply via email to