Hello Mauricio, Comments inline.
>+ >+IVSHMEM will not work with 2MB hugepages. It will work only with 1GB huge >pages. >Is this true? >I have used ivshmem with 2MB for a while without facing any problem. >AFAIK one can’t use the dpdk rings using 2MB pages when sharing with >VM. This is due to known limitation where sharing multiple file descriptors to >guest isn’t supported. For the same reason 1GB hugepages needs to be >allocated and exposed to the guest through ivshmem device as a single 1GB >page. >Do you mean, you could do the same with 2MB rings ? I would be interested >in the steps. > > >Yes, we use 2MB huge pages. What we did was to create a command line >generator for qemu, (https://github.com/netgroup-polito/un- >orchestrator/blob/master/orchestrator/compute_controller/plugins/kvm- >libvirt/cmdline_generator/cmdline_generator.c), it uses the IVHSMEM library >from DPDK to expose the rte_rings and the mempools to the guests. >It has to be used with a modified version of QEMU. There is a patch that >applies on qemu v2.2.1: (https://github.com/netgroup-polito/un- >orchestrator/blob/master/orchestrator/compute_controller/plugins/kvm- >libvirt/patches/ivshmem-qemu-2.2.1.patch) [BHANU]: well, I was not quite successful doing this with 2MB hugepages, but the same works with 1GB hugepages on the host. The cmdline_generator utility generated the below args, which I passed to qemu. Also note that I have patched Qemu v2.2.1 with 'ivshmem-qemu-2.2.1.patch'. " -device ivshmem,size=4M,shm=fd:/dev/hugepages/rtemap_0:0x0:0x200000:/dev/zero:0x0:0x1fc000:/var/run/.dpdk_ivshmem_metadata_cmdline:0x0:0x4000" Can you list the steps here? > >+ >+ The steps (1-5) in 3.3 section of INSTALL.DPDK guide will create & >initialize >DB, >+ start vswitchd and add dpdk devices to bridge br0. >+ >+ 1. Add DPDK ring port to the bridge >+ >+ ``` >+ ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr >+ ``` >+ >+ 2. Copy runtime configuration to VM, To achieve this copy the files to a >temporary >+ directory, say /tmp/rte_config and export the directory to the VM >+ >+ ``` >+ mkdir /tmp/rte_config >+ chmod 644 /tmp/rte_config >+ cp -a /run/.rte_config /run/.rte_hugepage_info /tmp/rte_config >+ ``` >+ >+ 3. Build modified Qemu >+ >+ ``` >+ cd /usr/src/ >+ wget https://github.com/01org/dpdk-ovs/archive/development.zip >+ unzip development.zip >+ cd dpdk-ovs-development/qemu >+ ./configure --target-list=x86_64-softmmu --enable-debug --extra- >cflags='-g' >+ make -j 4 >+ ``` >+ >+ 4. start Guest VM >+ >+ ``` >+ export VM_NAME=ivshmem-vm >+ export QCOW2_IMAGE=CentOS7_x86_64.qcow2 >+ export QEMU_BIN=/usr/src/dpdk-ovs-development/qemu/x86_64- >softmmu/qemu-system-x86_64 >+ >+ taskset 0x20 $QEMU_BIN -cpu host -smp 2,cores=2 -hda >$QCOW2_IMAGE -drive file=fat:rw:/tmp/rte_config,snapshot=off -m 4096M - >-enable-kvm -name $VM_NAME -nographic -vnc :2 -pidfile /tmp/vm1.pid - >mem-path /dev/hugepages -mem-prealloc -device >ivshmem,size=1024M,shm=fd:/dev/hugepages/rtemap_0:0x0:0x40000000 >+ ``` >+ >I am very curious about this way of sharing memory with a VM using ivshmem. >I haven't seen any similar thing before. >Why not to use the ivshmem library >(http://dpdk.org/doc/guides/prog_guide/ivshmem_lib.html) to generate the >qemu command line? >Yes, this can also be used to generate qemu command line. > >I think the best way to expose dpdkr ports to the guests is using the >IVSHMEM library, in this way many of the steps described on this guide are >not necessary. I could try to document it, the only problem I'm facing now if >the lack of time, then it would take some days. [BHANU] Any help in simplifying setting up the ivshmem would be highly appreciated. Can you share the steps so that I can incorporate in the next version and send it across (or) we can do that as an incremental patch once the refactored install guide is up streamed. Regards, Bhanu Prakash. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
