Hi Alex, Thank you for the quick response.
i have 370+G RAM in the system [root@localhost vcr]# cat /proc/meminfo MemTotal: 396231416 kB MemFree: 272301316 kB the reason i moved towards pci-e, sr-iov, vfio is primarily that reason. i can get upto 1.8Gbps iMix traffic performance on vhost-virtio-passthrough mode, i am looking for upwards to 4Gbps performance per vNIC. the error doesn't say what;s wrong,with my config. appreciate your help. -chintu- On Fri, May 27, 2016 at 11:49 AM, Alex Williamson < alex.william...@redhat.com> wrote: > On Fri, 27 May 2016 11:15:41 -0400 > chintu hetam <rometor...@gmail.com> wrote: > > > I have tried few things with libvirt, directly spawning VM with qemu-kvm > > even tried withe centos7 as a guest and in all cases vfio fails. > > > > here are the details!! > > > > 1) Device: Intel X710/XL710 > > 2) virt xml snapshot: > > > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0xaa' slot='0x02' function='0x0'/> > > </source> > > <alias name='hostdev0'/> > > <rom bar='on'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x09' > > function='0x0'/> > > </hostdev> > > <hostdev mode='subsystem' type='pci' managed='yes'> > > <driver name='vfio'/> > > <source> > > <address domain='0x0000' bus='0xaa' slot='0x06' function='0x0'/> > > </source> > > <alias name='hostdev1'/> > > <rom bar='on'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' > > function='0x0'/> > > </hostdev> > > > > 3) ip link show snapshot: > > 18: enp170s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop portid > > 00e0ed5d884c state DOWN mode DEFAULT group default qlen 1000 > > link/ether 00:e0:ed:5d:88:4c brd ff:ff:ff:ff:ff:ff > > vf 0 MAC aa:bb:cc:dd:ee:ff, spoof checking on, link-state auto > > vf 1 MAC aa:bb:cc:dd:ee:fe, spoof checking on, link-state auto > > vf 2 MAC aa:bb:cc:dd:ee:fd, spoof checking on, link-state auto > > vf 3 MAC aa:bb:cc:dd:ee:fc, spoof checking on, link-state auto > > 19: enp170s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop portid > > 00e0ed5d884d state DOWN mode DEFAULT group default qlen 1000 > > link/ether 00:e0:ed:5d:88:4d brd ff:ff:ff:ff:ff:ff > > vf 0 MAC aa:bb:cc:dd:ed:ff, spoof checking on, link-state auto > > vf 1 MAC aa:bb:cc:dd:ed:fe, spoof checking on, link-state auto > > vf 2 MAC aa:bb:cc:dd:ed:fd, spoof checking on, link-state auto > > vf 3 MAC aa:bb:cc:dd:ed:fc, spoof checking on, link-state auto > > > > 4) various system information > > [root@localhost vcr]# modinfo i40evf > > filename: > > > /lib/modules/4.4.6-300.fc23.x86_64/updates/drivers/net/ethernet/intel/i40evf/i40evf.ko > > version: 1.5.14 > > license: GPL > > description: Intel(R) 40-10 Gigabit Ethernet Virtual Function Driver > > author: Intel Corporation, <linux.n...@intel.com> > > srcversion: 522158BAB323575B03D48C8 > > alias: pci:v00008086d0000154Csv*sd*bc*sc*i* > > depends: > > vermagic: 4.4.6-300.fc23.x86_64 SMP mod_unload > > > > [root@localhost vcr]# modinfo i40e > > filename: > > > /lib/modules/4.4.6-300.fc23.x86_64/kernel/drivers/net/ethernet/intel/i40e/i40e.ko.xz > > version: 1.3.46-k > > license: GPL > > description: Intel(R) Ethernet Connection XL710 Network Driver > > author: Intel Corporation, <e1000-de...@lists.sourceforge.net> > > srcversion: A3FBA4AB3B91C574EA1FD38 > > alias: pci:v00008086d00001588sv*sd*bc*sc*i* > > alias: pci:v00008086d00001587sv*sd*bc*sc*i* > > alias: pci:v00008086d000037D2sv*sd*bc*sc*i* > > alias: pci:v00008086d000037D1sv*sd*bc*sc*i* > > alias: pci:v00008086d000037D0sv*sd*bc*sc*i* > > alias: pci:v00008086d00001587sv*sd*bc*sc*i* > > alias: pci:v00008086d00001589sv*sd*bc*sc*i* > > alias: pci:v00008086d00001586sv*sd*bc*sc*i* > > alias: pci:v00008086d00001585sv*sd*bc*sc*i* > > alias: pci:v00008086d00001584sv*sd*bc*sc*i* > > alias: pci:v00008086d00001583sv*sd*bc*sc*i* > > alias: pci:v00008086d00001581sv*sd*bc*sc*i* > > alias: pci:v00008086d00001580sv*sd*bc*sc*i* > > alias: pci:v00008086d0000157Fsv*sd*bc*sc*i* > > alias: pci:v00008086d00001574sv*sd*bc*sc*i* > > alias: pci:v00008086d00001572sv*sd*bc*sc*i* > > depends: ptp,vxlan > > intree: Y > > vermagic: 4.4.6-300.fc23.x86_64 SMP mod_unload > > parm: debug:Debug level (0=none,...,16=all) (int) > > > > However, in libvirt xml if i change driver to kvm it works but then as i > > understand it's e1000 equivalent and vfio is virtio performance > equivalent . > > Not remotely, in this context 'kvm' vs 'vfio' are both types of device > assignment, but kvm is deprecated here and will be going away at some > point (only legacy kvm device assignment, not the remainder of kvm). If > you want virtio performance equivalent, use virtio, PCI device > assignment offers better performance than virtio. > > > 5) lspci -tvv snapshot > > > > > > | +-03.0-[aa-ab]--+-00.0 Intel Corporation Ethernet > Controller > > X710 for 10GbE SFP+ > > | | +-00.1 Intel Corporation Ethernet > Controller > > X710 for 10GbE SFP+ > > | | +-00.2 Intel Corporation Ethernet > Controller > > X710 for 10GbE SFP+ > > | | +-00.3 Intel Corporation Ethernet > Controller > > X710 for 10GbE SFP+ > > | | +-02.0 Intel Corporation XL710/X710 Virtual > > Function > > | | +-02.1 Intel Corporation XL710/X710 Virtual > > Function > > | | +-02.2 Intel Corporation XL710/X710 Virtual > > Function > > | | +-02.3 Intel Corporation XL710/X710 Virtual > > Function > > | | +-06.0 Intel Corporation XL710/X710 Virtual > > Function > > | | +-06.1 Intel Corporation XL710/X710 Virtual > > Function > > | | +-06.2 Intel Corporation XL710/X710 Virtual > > Function > > | | \-06.3 Intel Corporation XL710/X710 Virtual > > Function > > > > 6) FreeBSD guest virtual function driver version is 1.2.11 > > > > 7) error log doesn't say much other than resource unavailable > > 2016-05-27 15:08:28.804+0000: starting up libvirt version: 1.2.18.2, > > package: 3.fc23 (Fedora Project, 2016-03-17-21:40:48, > > buildvm-22.phx2.fedoraproject.org), qemu version: 2.4.1 > (qemu-2.4.1-9.fc23) > > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin > > QEMU_AUDIO_DRV=spice /usr/bin/qemu-kvm -name generic -S -machine > > pc-i440fx-2.4,accel=kvm,usb=off,vmport=off,mem-merge=off -cpu SandyBridge > > -m 32768 -mem-prealloc -mem-path /dev/hugepages/libvirt/qemu -realtime > > mlock=on -smp 8,sockets=8,cores=1,threads=1 -uuid > > d8f61bc1-dd9a-470f-a26f-824e5e2b8359 -no-user-config -nodefaults -chardev > > > socket,id=charmonitor,path=/var/lib/libvirt/qemu/generic.monitor,server,nowait > > -mon chardev=charmonitor,id=monitor,mode=control -rtc > > base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet > > -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 > > -boot order=c,menu=on,strict=on -device > > ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device > > > ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 > > -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 > > -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 > > -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive > > > file=/home/vcr/Appliance-4.6.0.0_5-vtnet0.img,if=none,id=drive-virtio-disk0,format=raw > > -device > > > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-virtio-disk0,id=virtio-disk0 > > -drive > > > file=/dev/disk/by-id/scsi-36001405fdc0e99da3aa4b37b1403085c,if=none,id=drive-virtio-disk1,format=raw > > -device > > > virtio-blk-pci,scsi=off,bus=pci.0,addr=0xa,drive=drive-virtio-disk1,id=virtio-disk1 > > -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device > > > virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:30:a0:c0,bus=pci.0,addr=0x3 > > -chardev pty,id=charserial0 -device > > isa-serial,chardev=charserial0,id=serial0 -chardev > > spicevmc,id=charchannel0,name=vdagent -device > > > virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 > > -spice > > > port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on > > -device > > > qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 > > -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device > > hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev > > spicevmc,id=charredir0,name=usbredir -device > > usb-redir,chardev=charredir0,id=redir0 -chardev > > spicevmc,id=charredir1,name=usbredir -device > > usb-redir,chardev=charredir1,id=redir1 -device > > vfio-pci,host=aa:02.0,id=hostdev0,bus=pci.0,addr=0x9,rombar=1 -device > > vfio-pci,host=aa:06.0,id=hostdev1,bus=pci.0,addr=0xb,rombar=1 -device > > virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on > > char device redirected to /dev/pts/2 (label charserial0) > > qemu: qemu_thread_create: Resource temporarily unavailable > > 2016-05-27 15:09:20.735+0000: shutting down > > > > Any help will be appreciated. > > You're creating a 32G VM, how much memory does the host have? Anything > in dmesg? Thanks, > > Alex >
_______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
