Test Procedure with KVM guests + iPXE
=====================================
- 2 guests: iSCSI target/server and iSCSI initiator/client.
- 1 bridge for iSCSI traffic (virbr-iscsi, new), static ip.
- 1 bridge for internet access (virbr0, exists), dhcp ip.
- Release/Installer: Debian Buster Alpha 5 (netinst)
Host:
----
Configure the iSCSI bridge and QEMU access in the host:
# ip link add dev virbr-iscsi type bridge
# ip link set dev virbr-iscsi up
# echo 'allow virbr-iscsi' >>/etc/qemu/bridge.conf
iSCSI target:
------------
This guest serves an iSCSI target with one LUN
in iSCSI NIC with IP 10.0.0.1 for IP 10.0.0.2.
$ qemu-system-x86_64 \
-nodefaults \
-enable-kvm \
-smp 2 -m 4096 \
-serial stdio \
-nographic -vga none \
-netdev bridge,id=bridge-world,br=virbr0 \
-netdev bridge,id=bridge-iscsi,br=virbr-iscsi \
-device
virtio-net-pci,netdev=bridge-world,id=nic-world,mac=52:54:00:00:00:11
\
-device
virtio-net-pci,netdev=bridge-iscsi,id=nic-iscsi,mac=52:54:00:00:00:22
\
-drive file=debian-iscsi-target.qcow2,if=virtio \
-drive
file=debian-buster-DI-alpha5-amd64-netinst.iso,media=cdrom,read-only,if=scsi
# lsb_release -d
Description: Debian GNU/Linux buster/sid
Configure iSCSI NIC:
# cat <<EOF >/etc/network/interfaces.d/nic-iscsi
iface ens4 inet static
address 10.0.0.1/24
EOF
# ifup ens4
# ip addr list ens4
3: ens4: <...>
link/ether 52:54:00:00:00:22 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.1/24 brd 10.0.0.255 scope global ens4
...
Configure iSCSI target/lun:
# apt-get install -y tgt
# mkdir /var/lib/iscsi
# dd if=/dev/zero of=/var/lib/iscsi/disk bs=1 count=0 seek=8G
# tgtadm --lld iscsi --op new --mode target --tid 1 -T
iqn.2019-03.com.example:target1
# tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1
-b /var/lib/iscsi/disk
# tgtadm --lld iscsi --op bind --mode target --tid 1 -I 10.0.0.2
# tgt-admin --dump >/etc/tgt/conf.d/target1.conf
iSCSI initiator:
---------------
This guest first boots iPXE to configure iBFT,
and then boots/chainloads to debian-installer.
Later we install the patched disk-detect udeb.
(The netboot installer didn't find modules no
matter what, so this uses netinst iso's files)
$ wget http://boot.ipxe.org/ipxe.lkrn
$ cp
debian-buster-DI-alpha5-amd64-netinst.iso:install.amd/{vmlinuz,initrd.gz}
.
$ python3 -m http.server &
Serving HTTP on 0.0.0.0 port 8000 ...
$ qemu-system-x86_64 \
-nodefaults \
-enable-kvm \
-smp 2 -m 4096 \
-serial stdio \
-vga virtio \
-display vnc=0.0.0.0:1 \
-netdev bridge,id=bridge-world,br=virbr0 \
-netdev bridge,id=bridge-iscsi,br=virbr-iscsi \
-device
virtio-net-pci,netdev=bridge-world,id=nic-world,mac=52:54:00:00:00:01
\
-device
virtio-net-pci,netdev=bridge-iscsi,id=nic-iscsi,mac=52:54:00:00:00:02
\
-drive
file=debian-buster-DI-alpha5-amd64-netinst.iso,media=cdrom,read-only,if=scsi
\
-kernel ipxe.lkrn
Connect to VNC for iPXE shell:
$ vncviewer <address>:1
iPXE <...>
<wait for NIC address detection>
Press Ctrl-B for iPXE command line.
^B
iPXE>
Configure iSCSI NIC:
iPXE> ifopen net1
iPXE> set net1/ip 10.0.0.2
iPXE> set net1/netmask 255.255.255.0
Configure iBFT: (iSCSI portal 10.0.0.1, LUN 1 on target iqn.<...>:target1)
iPXE> sanhook iscsi:10.0.0.1:::1:iqn.2019-03.com.example:target1
Registered SAN device 0x80
Boot the installer
(add option 'disk-detect/ibft/enable=true' for installer
and option 'iscsi_auto' for system to boot with iBFT):
iPXE> ifopen net0
iPXE> kernel http://192.168.122.1:8000/vmlinuz initrd=initrd.gz
--- console=ttyS0
iPXE> initrd http://192.168.122.1:8000/initrd.gz
iPXE> boot
Back to serial console.
Proceed with the installer.
In 'Users and passwords' dialogs, select 'Go back', and 'Execute a
shell', and 'Continue'.
Bring up the iSCSI devices with iBFT
(manually or with patch in bug 924675)
~ # modprobe iscsi_ibft
~ # iscsistart -N
Setting up software interface ens5
~ # iscsistart -b
iscsistart: Logging into iqn.2019-03.com.example:target1 10.0.0.1:3260,1
iscsistart: can not connect to iSCSI daemon (111)!
iscsistart: version 2.0-874
iscsistart: Connection1:0 to [target:
iqn.2019-03.com.example:target1, portal: 10.0.0.1,3260] through
[iface: default] is operational now
~ # dmesg | grep -e iBFT -e sd
[ 0.015043] iBFT found at 0x9e520.
[ 59.892884] iBFT detected.
[ 67.254818] sd 3:0:0:1: Power-on or device reset occurred
[ 67.257239] sd 3:0:0:1: [sda] 16777216 512-byte logical blocks:
(8.59 GB/8.00 GiB)
[ 67.257242] sd 3:0:0:1: [sda] 4096-byte physical blocks
[ 67.257474] sd 3:0:0:1: [sda] Write Protect is off
[ 67.257476] sd 3:0:0:1: [sda] Mode Sense: 69 00 10 08
[ 67.257886] sd 3:0:0:1: [sda] Write cache: enabled, read cache:
enabled, supports DPO and FUA
[ 67.264334] sd 3:0:0:1: [sda] Attached SCSI disk
See interface 1 (ens4) is default interface in 192.168.122.0/24 range,
and interface 2 (ens5) is a iSCSI interface in 10.0.0.0/24 range.
~ # ip addr list
...
2: ens4: ...
link/ether 52:54:00:00:00:01 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.161/24 scope global ens4
...
3: ens5: ...
link/ether 52:54:00:00:00:02 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.2/24 brd 10.0.0.255 scope global ens5
...
Return with 'exit' and proceed with the installer until the 'Partition
disks' dialog.
(You should see the iSCSI LUN listed as 'SCSI4 (0,0,1) (sda) - 8.6 GB
IET VIRTUAL-DISK'.)
Test #0) Original partman-iscsi
----
In 'Partition disks' dialogs, select 'Go back', and 'Change debconf
priority', and 'Continue', and 'low'.
Return to the 'Partition disks' dialog.
Select 'Guided partitioning',
and 'Guided - use entire disk',
and 'SCSI4 (0,0,1) (sda) - 8.6 GB IET VIRTUAL-DISK,
and 'All files in one partition (recommended for new users)',
and 'Finish partitioning and write changes to disk',
and 'Yes' in 'Write the changes to disks?'.
This should partition and format the disk, and return to the menu, as
the debconf priority is 'low'.
Select 'Execute a shell', and 'Continue'.
See that the wrong network interface is used for the HWADDR field in
iscsi.initramfs,
and that the wrong network interface is set to manual (which disabled
DHCP on the default interface).
~ # cat /target/etc/iscsi/iscsi.initramfs
HWADDR="52:54:00:00:00:01"
ISCSI_TARGET_NAME="iqn.2019-03.com.example:target1"
ISCSI_TARGET_IP="10.0.0.1"
ISCSI_TARGET_PORT="3260"
ISCSI_TARGET_GROUP="1"
~ # cat /etc/network/interfaces
...
# The primary network interface
allow-hotplug ens4
iface ens4 inet manual
Fix /e/n/i ens4 back to DHCP:
~ # sed -i 's/\(iface ens4 inet\) manual/\1 dhcp/' /etc/network/interfaces
~ # cat /etc/network/interfaces
...
# The primary network interface
allow-hotplug ens4
iface ens4 inet dhcp
Test #1) Patched partman-iscsi, changes for patch 1/2
----
(use the iSCSI interface for HWADDR and /etc/network/interfaces)
Install the patched udeb:
~ # wget http://192.168.122.1:8000/partman-iscsi_60+ibft1_all.udeb
~ # udpkg --unpack partman-iscsi_60+ibft1_all.udeb
Verify the new option is not yet enabled.
~ # debconf-get partman-iscsi/iscsi_auto
false
Return with 'exit' and repeat the 'Partition disks'/'Execute a shell'
procedure from Test #0.
See that the iSCSI network interface is now used in HWADDR
and the default network interface still remains with DHCP enabled.
~ # cat /target/etc/iscsi/iscsi.initramfs
HWADDR="52:54:00:00:00:02"
ISCSI_TARGET_NAME="iqn.2019-03.com.example:target1"
ISCSI_TARGET_IP="10.0.0.1"
ISCSI_TARGET_PORT="3260"
ISCSI_TARGET_GROUP="1"
~ # cat /etc/network/interfaces
...
# The primary network interface
allow-hotplug ens4
iface ens4 inet dhcp
Test #2) Patched partman-iscsi, changes for patch 2/2
----
(use ISCSI_AUTO=true in /etc/iscsi/iscsi.initramfs)
Now enable the 'partman-iscsi/iscsi_auto' option,
and start a new debconf/menu to detect its value:
Install the patched udeb again, so the option re-appears:
~ # debconf-get partman-iscsi/iscsi_auto
~ #
~ # udpkg --unpack partman-iscsi_60+ibft1_all.udeb
~ # debconf-get partman-iscsi/iscsi_auto
false
~ # debconf-set partman-iscsi/iscsi_auto true
~ # debconf-get partman-iscsi/iscsi_auto
true
~ # debconf -o d-i /usr/bin/main-menu
Repeat the 'Partition disks'/'Execute a shell' procedure from Test #0.
See that 'ISCSI_AUTO=true' is not configured in 'iscsi.initramfs'.
And the default interface still remains configured with DHCP (no regression).
~ # cat /target/etc/iscsi/iscsi.initramfs
ISCSI_AUTO=true
~ # cat /etc/network/interfaces
...
# The primary network interface
allow-hotplug ens4
iface ens4 inet dhcp
Return with 'exit', then 'Change debconf priority' to 'high' again,
and proceed/finish the installation.
System reboots.
Back to VNC console, go again to iPXE shell:
$ vncviewer <address>:1
iPXE <...>
<wait for NIC address detection>
Press Ctrl-B for iPXE command line.
^B
iPXE>
Configure iSCSI NIC:
iPXE> ifopen net1
iPXE> set net1/ip 10.0.0.2
iPXE> set net1/netmask 255.255.255.0
Boot from iSCSI (iSCSI portal 10.0.0.1, LUN 1 on target iqn.<...>:target1)
iPXE> sanboot iscsi:10.0.0.1:::1:iqn.2019-03.com.example:target1
Registered SAN device 0x80
Booting from SAN device 0x80
Back to serial console.
Watch the system boot with ISCSI_AUTO=true:
GNU GRUB version 2.02+dfsg1-12
+----------------------------------------------------------------------------+
|*Debian GNU/Linux
|
| Advanced options for Debian GNU/Linux
|
|
|
<...>
The highlighted entry will be executed automatically in 0s.
Loading Linux 4.19.0-2-amd64 ...
Loading initial ramdisk ...
Setting up software interface ens5
iscsistart: Logging into iqn.2019-03.com.example:target1 10.0.0.1:3260,1
iscsistart: version 2.0-874
iscsistart: Connection1:0 to [target:
iqn.2019-03.com.example:target1, portal: 10.0.0.1,3260] through
[iface: default] is operational now
/dev/sda1: clean, 35475/262144 files, 364064/1048064 blocks
Debian GNU/Linux buster/sid debian-iscsi-initiator ttyS0
debian-iscsi-initiator login: root
Password:
root@debian-iscsi-initiator:~# mount | grep -w /
/dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro)
root@debian-iscsi-initiator:~# ls -ld /sys/block/sda
<...> /sys/block/sda ->
../devices/platform/host3/session1/target3:0:0/3:0:0:1/block/sda
root@debian-iscsi-initiator:~# cat
/sys/devices/platform/host3/session1/iscsi_session/session1/targetname
iqn.2019-03.com.example:target1
root@debian-iscsi-initiator:~# cat /etc/iscsi/iscsi.initramfs
ISCSI_AUTO=true
# cat /etc/network/interfaces
...
# The primary network interface
allow-hotplug ens4
iface ens4 inet dhcp
root@debian-iscsi-initiator:~# ip addr list
...
2: ens4: ...
link/ether 52:54:00:00:00:01 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.162/24 brd 192.168.122.255 scope global dynamic ens4
...
3: ens5: ...
link/ether 52:54:00:00:00:02 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.2/24 brd 10.0.0.255 scope global ens5
...
--
Mauricio Faria de Oliveira
