Re: [dpdk-dev] Deadlock when start virtio_user + vhost_kernel

[Jason Wang]

On 2018年09月12日 12:27, Tiwei Bie wrote:

On Wed, Sep 12, 2018 at 11:47:20AM +0800, Jason Wang wrote:

Hi:

Try to launch virtio_user + vhost_kernel with: testpmd
--vdev=virtio_user0,path=/dev/vhost-net -- -i

It seems we get a deadlock on
rte_rwlock_read_lock(&mcfg->memory_hotplug_lock)

Yes, you're right. There is a deadlock here.
FYI, it can be fixed by below patch:
http://patches.dpdk.org/patch/44290/

Thanks for the pointer, this fixes the deadlock.

calltrace:

Thread 1 "testpmd" received signal SIGINT, Interrupt.
rte_memseg_contig_walk (func=func@entry=0x555555a5e630 <add_memory_region>,
arg=arg@entry=0x7fffffffcec0)
     at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
469 rte_rwlock_read_lock(&mcfg->memory_hotplug_lock);
(gdb) bt
#0  rte_memseg_contig_walk (func=func@entry=0x555555a5e630
<add_memory_region>, arg=arg@entry=0x7fffffffcec0)
     at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
#1  0x0000555555a5e9b1 in prepare_vhost_memory_kernel () at
/home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:118
#2  vhost_kernel_ioctl (dev=0x7ffbf5fb3300, req=<optimized out>,
arg=<optimized out>)
     at
/home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:190
#3  0x0000555555a5f211 in virtio_user_mem_event_cb (type=<optimized out>,
addr=<optimized out>, len=<optimized out>, arg=0x7ffbf5fb3300)
     at
/home/devel/git/dpdk/drivers/net/virtio/virtio_user/virtio_user_dev.c:297
#4  0x000055555574814b in eal_memalloc_mem_event_notify
(event=event@entry=RTE_MEM_EVENT_ALLOC, start=start@entry=0x7ffbf6000000,
     len=len@entry=94371840) at
/home/devel/git/dpdk/lib/librte_eal/common/eal_common_memalloc.c:248
#5  0x00005555557563f6 in try_expand_heap_primary (contig=false, bound=0,
align=64, flags=0, socket=0, elt_size=0, pg_sz=<optimized out>,
     heap=0x7ffff7ff667c) at
/home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:344
#6  try_expand_heap (heap=heap@entry=0x7ffff7ff667c, pg_sz=<optimized out>,
elt_size=elt_size@entry=92403968, socket=socket@entry=0,
     flags=flags@entry=0, align=align@entry=64, bound=0, contig=false) at
/home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:426
#7  0x0000555555756928 in alloc_more_mem_on_socket
(heap=heap@entry=0x7ffff7ff667c, size=size@entry=92403968,
socket=socket@entry=0,
     flags=flags@entry=0, align=align@entry=64, bound=bound@entry=0,
contig=false) at
/home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:554
#8  0x0000555555756e37 in heap_alloc_on_socket (contig=false, bound=0,
align=64, flags=0, socket=0, size=92403968, type=<optimized out>)
     at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:590
#9  malloc_heap_alloc (type=<optimized out>, size=92403968,
socket_arg=<optimized out>, flags=0, align=<optimized out>, bound=0,
contig=false)
     at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:626
#10 0x0000555555753fc1 in rte_zmalloc () at
/home/devel/git/dpdk/lib/librte_eal/common/rte_malloc.c:74
#11 0x00005555556192f9 in init_port () at
/home/devel/git/dpdk/app/test-pmd/testpmd.c:2645
#12 main (argc=4, argv=0x7fffffffdb18) at
/home/devel/git/dpdk/app/test-pmd/testpmd.c:2734

And I also get this warning:

vhost_kernel_ioctl(): VHOST_SET_OWNER failed: Device or resource busy

Since below commit:
bce7e9050f9b ("net/virtio-user: fix start with kernel vhost")
https://github.com/DPDK/dpdk/commit/bce7e9050f9b

The vhost SET_OWNER will be done in virtio_user_start_device()
unconditionally. It caused above harmless but annoying warning.

Ok, but it's better to remove it anyway in the future.

Thanks

Both looks like a bug that needs to be fixed.

Thanks