When using qemu, /dev/tunX interfaces are being created with the wrong
permissions. It's not a /dev/MAKEDEV problem
# cd /dev && ./MAKEDEV tun4
works just peachy. The problem is when /dev/tunX is created on the fly.
...Are /dev/tunX created on the fly?
hmmm... maybe that happens when I run my qemu command?
sudo -C 4 -u qemu /usr/local/bin/qemu -hda
/home/qemu/4.4/i386/stable/stable_44_i386.img -m 256 -net
nic,model=rtl8139,macaddr=52:54:00:c7:44:01 -net tap,fd=3 -nographic
-localtime -no-fd-bootchk -pidfile /home/qemu/4.4/i386/stable/qemu.pid
3<>/dev/tun10
Perhaps I should be asking for a reminder in
/usr/local/share/doc/qemu/README.OpenBSD that tun(4) interfaces, other
than tun0-3, need to be created with MAKEDEV?
Don't know.
Hope this helps someone else.
BACKGROUND:
I'm having trouble with qemu and tun(4) interfaces. It seems I can
only use tun0, tun1, tun2, and tun3.
My goal is to run multiple instances of qemu on a dedicated server for
testing things, hosting web applications, building -stable, stuff like
that.
I have 11 virtual machines that will each work properly as long as it
is using one of tun0, tun1, tun2, or tun3. Each virtual machine has
its own distinct IP and MAC address.
Today I attempted to run them all at once using tun0 through tun10.
The result was as such:
>From a separate computer, I could ping virtual machines that used
tun0-3, but not tun4-10. arping gave the same results.
I ran through the virtual machines again using arping, and listened
with tcpdump on my qemu server.
sudo tcpdump -n -i tun0
- could see requests and replies to the vm on tun0
- did not see requests or replies to vms on tun1-3
- could see requests to vms on tun4-10
At this point it has occurred to me that perhaps /dev/tun[4-10] do not exist:
q...@qemu $ ls -l /dev/tun*
crw------- 1 root wheel 40, 0 Aug 23 16:36 /dev/tun0
crw------- 1 root wheel 40, 1 Aug 23 16:36 /dev/tun1
-rw-r--r-- 1 root wheel 0 Aug 23 09:49 /dev/tun10
-rw-r--r-- 1 root wheel 5299 Apr 13 08:38 /dev/tun101
crw------- 1 root wheel 40, 2 Aug 23 16:31 /dev/tun2
crw------- 1 root wheel 40, 3 Aug 23 16:36 /dev/tun3
-rw-r--r-- 1 root wheel 3594 Aug 23 16:35 /dev/tun4
-rw-r--r-- 1 root wheel 450 Aug 23 09:47 /dev/tun5
-rw-r--r-- 1 root wheel 1506 Aug 23 16:35 /dev/tun6
-rw-r--r-- 1 root wheel 120 Aug 23 09:46 /dev/tun8
-rw-r--r-- 1 root wheel 120 Aug 23 09:45 /dev/tun9
but they do. Although they seem to have not been created properly.
So I decided to get rid of them 'sudo rm /dev/tun4' [...] tun101, now
I've got:
q...@qemu $ ls -l /dev/tun*
crw------- 1 root wheel 40, 0 Aug 23 16:45 /dev/tun0
crw------- 1 root wheel 40, 1 Aug 23 16:44 /dev/tun1
crw------- 1 root wheel 40, 2 Aug 23 16:31 /dev/tun2
crw------- 1 root wheel 40, 3 Aug 23 16:45 /dev/tun3
So far so good... just make some new devices:
q...@qemu $ cd /dev
q...@qemu $ sudo ./MAKEDEV tun4
[...]
q...@qemu $ sudo ./MAKEDEV tun7
q...@qemu $ ls -l /dev/tun*
crw------- 1 root wheel 40, 0 Aug 23 16:45 /dev/tun0
crw------- 1 root wheel 40, 1 Aug 23 16:44 /dev/tun1
crw------- 1 root wheel 40, 2 Aug 23 16:31 /dev/tun2
crw------- 1 root wheel 40, 3 Aug 23 16:45 /dev/tun3
crw------- 1 root wheel 40, 4 Aug 23 16:58 /dev/tun4
crw------- 1 root wheel 40, 5 Aug 23 16:59 /dev/tun5
crw------- 1 root wheel 40, 6 Aug 23 16:59 /dev/tun6
crw------- 1 root wheel 40, 7 Aug 23 16:59 /dev/tun7
Now I'm wondering how those devices got jacked up in the first place,
decided to stop and reboot to see if tun[8-10] are created by /etc/rc.
Nope.
Started the vm using tun6: woohoo! works slicker 'n snot!
Started the vm that uses tun8, now I see:
q...@qemu $ ls -l /dev/tun*
crw------- 1 root wheel 40, 0 Aug 23 16:45 /dev/tun0
crw------- 1 root wheel 40, 1 Aug 23 16:44 /dev/tun1
crw------- 1 root wheel 40, 2 Aug 23 16:31 /dev/tun2
crw------- 1 root wheel 40, 3 Aug 23 16:45 /dev/tun3
crw------- 1 root wheel 40, 4 Aug 23 16:58 /dev/tun4
crw------- 1 root wheel 40, 5 Aug 23 16:59 /dev/tun5
crw------- 1 root wheel 40, 6 Aug 23 16:59 /dev/tun6
crw------- 1 root wheel 40, 7 Aug 23 16:59 /dev/tun7
-rw-r--r-- 1 root wheel 0 Aug 23 17:05 /dev/tun8
Yeah, that's not right.
Some more information:
Running a snapshot from August 16th.
q...@qemu $ dmesg |head -1
OpenBSD 4.8 (GENERIC.MP) #335: Mon Aug 16 09:09:20 MDT 2010
q...@qemu $ pkg_info |grep qemu
qemu-0.12.4p0 multi system emulator
qemu commands:
sudo -C 4 -u qemu /usr/local/bin/qemu -hda
/home/qemu/4.4/i386/stable/stable_44_i386.img -m 256 -net
nic,model=rtl8139,macaddr=52:54:00:c7:44:01 -net tap,fd=3 -nographic
-localtime -no-fd-bootchk -pidfile /home/qemu/4.4/i386/stable/qemu.pid
3<>/dev/tun10
sudo -C 4 -u qemu /usr/local/bin/qemu -hda
/home/qemu/4.5/i386/stable/stable_45_i386.img -m 256 -net
nic,model=rtl8139,macaddr=52:54:00:c7:45:02 -net tap,fd=3 -nographic
-localtime -no-fd-bootchk -pidfile /home/qemu/4.5/i386/stable/qemu.pid
3<>/dev/tun9
sudo -C 4 -u qemu /usr/local/bin/qemu -hda
/home/qemu/4.6/i386/stable/stable_46_i386.img -m 256 -net
nic,model=rtl8139,macaddr=52:54:00:c7:46:02 -net tap,fd=3 -nographic
-localtime -no-fd-bootchk -pidfile /home/qemu/4.6/i386/stable/qemu.pid
3<>/dev/tun5
sudo -C 4 -u qemu /usr/local/bin/qemu -hda
/home/qemu/4.7/i386/stable/stable_47_i386.img -m 256 -net
nic,model=rtl8139,macaddr=52:54:00:c7:47:02 -net tap,fd=3 -nographic
-localtime -no-fd-bootchk -pidfile /home/qemu/4.7/i386/stable/qemu.pid
3<>/dev/tun1
sudo -C 4 -u qemu /usr/local/bin/qemu-system-x86_64 -hda
/home/qemu/4.5/amd64/stable/stable_45_amd64.img -m 256 -net
nic,model=rtl8139,macaddr=52:54:00:c7:45:01 -net tap,fd=3 -nographic
-localtime -no-fd-bootchk -pidfile
/home/qemu/4.5/amd64/stable/qemu.pid 3<>/dev/tun8
sudo -C 4 -u qemu /usr/local/bin/qemu-system-x86_64 -hda
/home/qemu/4.6/amd64/stable/stable_46_amd64.img -m 256 -net
nic,model=rtl8139,macaddr=52:54:00:c7:46:01 -net tap,fd=3 -nographic
-localtime -no-fd-bootchk -pidfile
/home/qemu/4.6/amd64/stable/qemu.pid 3<>/dev/tun4
sudo -C 4 -u qemu /usr/local/bin/qemu-system-x86_64 -hda
/home/qemu/4.7/amd64/stable/stable_47_amd64.img -m 256 -net
nic,model=rtl8139,macaddr=52:54:00:c7:47:01 -net tap,fd=3 -nographic
-localtime -no-fd-bootchk -pidfile
/home/qemu/4.7/amd64/stable/qemu.pid 3<>/dev/tun0
q...@qemu $ cat /etc/hostname.bridge0
add tun0
add tun1
add tun2
add tun3
add tun4
add tun5
add tun6
add tun7
add tun8
add tun9
add tun10
add bnx1
up
hostname.tun0 through hostname .tun10 are all the same.
q...@qemu $ cat /etc/hostname.tun0
link0
up
q...@qemu $ ifconfig bridge0
bridge0: flags=41<UP,RUNNING>
groups: bridge
priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp
designated: id 00:00:00:00:00:00 priority 0
bnx1 flags=3<LEARNING,DISCOVER>
port 4 ifpriority 0 ifcost 0
tun10 flags=3<LEARNING,DISCOVER>
port 7 ifpriority 0 ifcost 0
tun9 flags=3<LEARNING,DISCOVER>
port 15 ifpriority 0 ifcost 0
tun8 flags=3<LEARNING,DISCOVER>
port 14 ifpriority 0 ifcost 0
tun7 flags=3<LEARNING,DISCOVER>
port 13 ifpriority 0 ifcost 0
tun6 flags=3<LEARNING,DISCOVER>
port 12 ifpriority 0 ifcost 0
tun5 flags=3<LEARNING,DISCOVER>
port 11 ifpriority 0 ifcost 0
tun4 flags=3<LEARNING,DISCOVER>
port 10 ifpriority 0 ifcost 0
tun3 flags=3<LEARNING,DISCOVER>
port 9 ifpriority 0 ifcost 0
tun2 flags=3<LEARNING,DISCOVER>
port 8 ifpriority 0 ifcost 0
tun1 flags=3<LEARNING,DISCOVER>
port 6 ifpriority 0 ifcost 0
tun0 flags=3<LEARNING,DISCOVER>
port 5 ifpriority 0 ifcost 0
Addresses (max cache: 100, timeout: 240):
00:26:2d:10:f7:93 bnx1 0 flags=0<>
00:50:04:61:82:e8 bnx1 1 flags=0<>
[...]
q...@qemu $ sudo grep -v -e'^$' -e'^#' /etc/pf.conf
set skip on lo
pass # to establish keep-state
block in on ! lo0 proto tcp to port 6000:6010
