Thank you both, that fixed my issue. Note that when using the config from https://fdio-vpp.readthedocs.io/en/latest/usecases/vhost/xmlexample.html#:~:text=%3Ccell%20id%3D%270%27%20cpus%3D%270%27%20memory%3D%27262144%27%20unit%3D%27KiB%27%20memAccess%3D%27shared%27/%3E it results in a kernel panic when the VM boots. I noticed that libvirt is actually aligning the <numa><cell id='0' cpus='0' memory='262144' unit='KiB' memAccess='shared'/></numa> to the <memory unit='KiB'>262144</memory> and <currentMemory unit='KiB'>262144</currentMemory> config. Hence using the parameters provided from the doc as such doesn't work.
Using: <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <numa><cell id='0' cpus='0' memory='4194304' unit='KiB' memAccess='shared'/></numa> allowed the VM to start and traffic passing through. Pretty sure that some tuning can be done using the documentation provided by Xiaodong. Thanks again Benjamin The content of this email is confidential and intended for the recipient specified in the message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future. On Thu, Mar 2, 2023 at 9:06 PM steven luong via lists.fd.io <sluong= cisco....@lists.fd.io> wrote: > It is likely that you are missing memAccess=’shared’ > > > > > https://fdio-vpp.readthedocs.io/en/latest/usecases/vhost/xmlexample.html#:~:text=%3Ccell%20id%3D%270%27%20cpus%3D%270%27%20memory%3D%27262144%27%20unit%3D%27KiB%27%20memAccess%3D%27shared%27/%3E > > > > *From: *<vpp-dev@lists.fd.io> on behalf of Benjamin Vandendriessche < > b...@openccnt.com> > *Reply-To: *"vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io> > *Date: *Thursday, March 2, 2023 at 9:24 AM > *To: *"vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io> > *Subject: *[vpp-dev] Vhost-user interface not working > > > > Hello, > > > > I followed the guide here > https://fdio-vpp.readthedocs.io/en/latest/usecases/vhost/vhost.html and > get into an issue as the VM running over libvirt (6.0.0) is not receiving > the traffic from the bare metal machine. > > I'm using Ubuntu 20.04.3 LTS and I run VPP version: > > > > vpp# show ver > vpp v22.06.1-release built by root on bv at 2023-03-02T14:39:16 > > > > The XML of the VM: > > > > <domain type='kvm'> > <name>vm1</name> > <uuid>32dede65-6db7-4db4-9093-f6484ea40836</uuid> > <memory unit='KiB'>1048576</memory> > <currentMemory unit='KiB'>1048576</currentMemory> > <memoryBacking> > <hugepages/> > </memoryBacking> > <vcpu placement='static'>2</vcpu> > <os> > <type arch='x86_64' machine='pc-i440fx-focal'>hvm</type> > <boot dev='hd'/> > </os> > <features> > <acpi/> > <apic/> > </features> > <cpu mode='host-model' check='partial'/> > <clock offset='utc'> > <timer name='rtc' tickpolicy='catchup'/> > <timer name='pit' tickpolicy='delay'/> > <timer name='hpet' present='no'/> > </clock> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>destroy</on_crash> > <pm> > <suspend-to-mem enabled='no'/> > <suspend-to-disk enabled='no'/> > </pm> > <devices> > <emulator>/usr/bin/qemu-system-x86_64</emulator> > <disk type='file' device='disk'> > <driver name='qemu' type='raw'/> > <source file='/home/bv/ubuntu'/> > <target dev='hda' bus='ide'/> > <address type='drive' controller='0' bus='0' target='0' unit='0'/> > </disk> > <disk type='file' device='cdrom'> > <driver name='qemu' type='raw'/> > <target dev='hdb' bus='ide'/> > <readonly/> > <address type='drive' controller='0' bus='0' target='0' unit='1'/> > </disk> > <controller type='usb' index='0' model='ich9-ehci1'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x7'/> > </controller> > <controller type='usb' index='0' model='ich9-uhci1'> > <master startport='0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x0' multifunction='on'/> > </controller> > <controller type='usb' index='0' model='ich9-uhci2'> > <master startport='2'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x1'/> > </controller> > <controller type='usb' index='0' model='ich9-uhci3'> > <master startport='4'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x2'/> > </controller> > <controller type='pci' index='0' model='pci-root'/> > <controller type='ide' index='0'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x01' > function='0x1'/> > </controller> > <interface type='network'> > <mac address='52:54:00:c6:99:a2'/> > <source network='default'/> > > <model type='e1000'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x0'/> > </interface> > <interface type='vhostuser'> > <mac address='52:54:00:4c:47:f2'/> > <source type='unix' path='/tmp//vm00.sock' mode='server'/> > <model type='virtio'/> > <driver rx_queue_size='1024' tx_queue_size='1024'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x06' > function='0x0'/> > </interface> > <serial type='pty'> > <target type='isa-serial' port='0'> > <model name='isa-serial'/> > </target> > </serial> > <console type='pty'> > <target type='serial' port='0'/> > </console> > <input type='tablet' bus='usb'> > <address type='usb' bus='0' port='1'/> > </input> > <input type='mouse' bus='ps2'/> > <input type='keyboard' bus='ps2'/> > <graphics type='vnc' port='5901' autoport='no' listen='0.0.0.0'> > <listen type='address' address='0.0.0.0'/> > </graphics> > <video> > <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' > primary='yes'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x0'/> > </video> > <memballoon model='virtio'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' > function='0x0'/> > </memballoon> > </devices> > </domain> > > > > I can see the vhost interface has been properly setup: > > > > vpp# sh vhost-user > Virtio vhost-user interfaces > Global: > coalesce frames 32 time 1e-3 > Number of rx virtqueues in interrupt mode: 0 > Number of GSO interfaces: 0 > Thread 0: Polling queue count 1 > Interface: VirtualEthernet0/0/0 (ifindex 2) > Number of qids 2 > virtio_net_hdr_sz 12 > features mask (0xfffffffbdfffa27c): > features (0x150208000): > VIRTIO_NET_F_MRG_RXBUF (15) > VIRTIO_NET_F_GUEST_ANNOUNCE (21) > VIRTIO_RING_F_INDIRECT_DESC (28) > VHOST_USER_F_PROTOCOL_FEATURES (30) > VIRTIO_F_VERSION_1 (32) > protocol features (0x3) > VHOST_USER_PROTOCOL_F_MQ (0) > VHOST_USER_PROTOCOL_F_LOG_SHMFD (1) > > socket filename /tmp/vm00.sock type client errno "Success" > > rx placement: > thread 0 on vring 1, polling > tx placement > threads 0 on vring 0: lock-free > > Memory regions (total 1) > region fd guest_phys_addr memory_size userspace_addr > mmap_offset mmap_addr > ====== ===== ================== ================== ================== > ================== ================== > 0 192 0x0000000000000000 0x0000000040000000 0x00007fb34be00000 > 0x0000000000000000 0x00007f0fb0000000 > > Virtqueue 0 (TX) > global TX queue index 3 > qsz 1024 last_avail_idx 0 last_used_idx 0 last_kick 0 > avail.flags 0 avail event idx 0 avail.idx 0 used.flags 1 used event idx > 0 used.idx 0 > kickfd 193 callfd 194 errfd -1 > > Virtqueue 1 (RX) > global RX queue index 3 > qsz 1024 last_avail_idx 0 last_used_idx 0 last_kick 0 > avail.flags 0 avail event idx 0 avail.idx 0 used.flags 1 used event idx > 0 used.idx 0 > kickfd 190 callfd 195 errfd -1 > > > > And all interfaces are UP: > > > > vpp# sh int > Name Idx State MTU (L3/IP4/IP6/MPLS) > Counter Count > TenGigabitEthernet44/0/0 1 up 9000/0/0/0 rx > packets 754 > rx > bytes 45240 > drops > 144 > VirtualEthernet0/0/0 2 up 9000/0/0/0 tx > packets 610 > tx > bytes 36600 > drops > 610 > > tx-error 144 > local0 0 down 0/0/0/0 > > > > However I see errors happening with VirtualEthernet0/0/0 Tx: > > > > vpp# show errors > Count Node Reason > Severity > 784 l2-output L2 output packets > error > 784 l2-learn L2 learn packets > error > 1 l2-learn L2 learn misses > error > 784 l2-input L2 input packets > error > 784 l2-flood L2 flood packets > error > 640 VirtualEthernet0/0/0-tx tx packet drops (no available > descr error > 144 VirtualEthernet0/0/0-output interface is down > error > > > > I've been trying with multiple VM and got the same result. > > > > How can I troubleshoot this issue ? > > > > Thanks, > > Best regards, > > Benjamin > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#22660): https://lists.fd.io/g/vpp-dev/message/22660 Mute This Topic: https://lists.fd.io/mt/97343942/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
