On 12.07.2016 05:43, Yuanhan Liu wrote:
> On Mon, Jul 11, 2016 at 02:47:56PM +0300, Ilya Maximets wrote:
>> On 11.07.2016 14:05, Yuanhan Liu wrote:
>>> On Mon, Jul 11, 2016 at 12:50:24PM +0300, Ilya Maximets wrote:
>>>> On 11.07.2016 11:38, Yuanhan Liu wrote:
>>>>> On Sun, Jul 10, 2016 at 09:17:31PM +0800, Yuanhan Liu wrote:
>>>>>> On Fri, Jul 08, 2016 at 02:48:56PM +0300, Ilya Maximets wrote:
>>>>>>>
>>>>>>> Another point is that crash constantly happens on queue_id=3 (second RX
>>>>>>> queue) in
>>>>>>> my scenario. It is newly allocated virtqueue while reconfiguration from
>>>>>>> rxq=1 to
>>>>>>> rxq=2.
>>>>>>
>>>>>> That's a valuable message: what's your DPDK HEAD commit while triggering
>>>>>> this issue?
>>>>
>>>> fbfd99551ca3 ("mbuf: add raw allocation function")
>>>>
>>>>>
>>>>> I guess I have understood what goes wrong in you case.
>>>>>
>>>>> I would guess that your vhost has 2 queues (here I mean queue-pairs,
>>>>> including one Tx and Rx queue; below usage is the same) configured,
>>>>> so does to your QEMU. However, you just enabled 1 queue while starting
>>>>> testpmd inside the guest, and you want to enable 2 queues by running
>>>>> following testpmd commands:
>>>>>
>>>>> stop
>>>>> port stop all
>>>>> port config all rxq 2
>>>>> port config all txq 2
>>>>> port start all
>>>>>
>>>>> Badly, that won't work for current virtio PMD implementation, and what's
>>>>> worse, it triggers a vhost crash, the one you saw.
>>>>>
>>>>> Here is how it comes. Since you just enabled 1 queue while starting
>>>>> testpmd, it will setup 1 queue only, meaning only one queue's **valid**
>>>>> information will be sent to vhost. You might see SET_VRING_ADDR
>>>>> (and related vhost messages) for the other queue as well, but they
>>>>> are just the dummy messages: they don't include any valid/real
>>>>> information about the 2nd queue: the driver don't setup it after all.
>>>>>
>>>>> So far, so good. It became broken when you run above commands. Those
>>>>> commands do setup for the 2nd queue, however, they failed to trigger
>>>>> the QEMU virtio device to start the vhost-user negotiation, meaning
>>>>> no SET_VRING_ADDR will be sent for the 2nd queue, leaving vhost
>>>>> untold and not updated.
>>>>>
>>>>> What's worse, above commands trigger the QEMU to send SET_VRING_ENABLE
>>>>> messages, to enable all the vrings. And since the vrings for the 2nd
>>>>> queue are not properly configured, the crash happens.
>>>>
>>>> Hmm, why 2nd queue works properly with my fix to vhost in this case?
>>>
>>> Hmm, really? You are sure that data flows in your 2nd queue after those
>>> commands? From what I know is that your patch just avoid a crash, but
>>> does not fix it.
>>
>> Oh, sorry. Yes, it doesn't work. With my patch applied I have a QEMU hang.
>
> The crash actually could be avoided by commit 0823c1cb0a73 ("vhost:
> workaround stale vring base"), accidentally. That's why I asked you
> above the HEAD commit you were using.
Thanks for pointing this. I'll check it.
>>>>> So maybe we should do virtio reset on port start?
>>>>
>>>> I guess it was removed by this patch:
>>>> a85786dc816f ("virtio: fix states handling during initialization").
>>>
>>> Seems yes.
>
> Actually, we should not do that: do reset on port start. The right fix
> should be allocating MAX queues virtio device supports (2 here). This
> would allow us changing the queue number dynamically.
Yes, I agree that this is the right way to fix this issue.
> But this doesn't sound a simple fix; it involves many code changes, due
> to it was not designed this way before. Therefore, we will not fix it
> in this release, due to it's too late. Let's fix it in the next release
> instead. For the crash issue, it will not happen with the latest HEAD.
> Though it's accident fix, I think we are fine here.
>
> --yliu
>
>
