On 11/1/19 8:16 AM, Liu, Yong wrote:
>
>
>> -----Original Message-----
>> From: Adrian Moreno [mailto:amore...@redhat.com]
>> Sent: Friday, November 01, 2019 1:48 AM
>> To: Liu, Yong <yong....@intel.com>; Bie, Tiwei <tiwei....@intel.com>
>> Cc: maxime.coque...@redhat.com; Wang, Zhihong <zhihong.w...@intel.com>;
>> dev@dpdk.org
>> Subject: Re: [PATCH v3] vhost: fix vhost user virtqueue not accessible
>>
>> Hi Marvin,
>>
>> On 10/31/19 3:54 PM, Liu, Yong wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Bie, Tiwei
>>>> Sent: Thursday, October 31, 2019 6:42 PM
>>>> To: Liu, Yong <yong....@intel.com>
>>>> Cc: maxime.coque...@redhat.com; Wang, Zhihong <zhihong.w...@intel.com>;
>>>> amore...@redhat.com; dev@dpdk.org
>>>> Subject: Re: [PATCH v3] vhost: fix vhost user virtqueue not accessible
>>>>
>>>> On Wed, Oct 30, 2019 at 10:56:02PM +0800, Marvin Liu wrote:
>>>>> Log feature is disabled in vhost user, so that log address was invalid
>>>>> when checking. Check whether log address is valid can workaround it.
>>>>> Also log address should be translated in packed ring virtqueue.
>>>>>
>>>>> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
>>>>>
>>>>> Signed-off-by: Marvin Liu <yong....@intel.com>
>>>>> ---
>>>>> lib/librte_vhost/vhost_user.c | 30 +++++++++++++-----------------
>>>>> 1 file changed, 13 insertions(+), 17 deletions(-)
>>>>>
>>>>> diff --git a/lib/librte_vhost/vhost_user.c
>>>> b/lib/librte_vhost/vhost_user.c
>>>>> index 61ef699ac..7754d2467 100644
>>>>> --- a/lib/librte_vhost/vhost_user.c
>>>>> +++ b/lib/librte_vhost/vhost_user.c
>>>>> @@ -641,11 +641,23 @@ translate_ring_addresses(struct virtio_net *dev,
>>>> int vq_index)
>>>>> struct vhost_vring_addr *addr = &vq->ring_addrs;
>>>>> uint64_t len, expected_len;
>>>>>
>>>>> + dev = numa_realloc(dev, vq_index);
>>>>
>>>> We need to update `vq->desc` first before doing numa_realloc.
>>>>
>> https://github.com/DPDK/dpdk/blob/19397c7bf2545e6adab41b657a1f1da3c7344e7b/
>>>> lib/librte_vhost/vhost_user.c#L445
>>>>
>>>>> + vq = dev->virtqueue[vq_index];
>>>>> + if (addr->flags & (1 << VHOST_VRING_F_LOG)) {
>>>>
>> I fear the possible consequences of this change.
>> Before 04cfc7fdbfca the approach was "best-effort". The log address would
>> be
>> assigned without further checks:
>>
>> vq->log_guest_addr = addr->log_guest_addr;
>>
>> Then, the behavior changed and an error was generated if the log address
>> was
>> invalid, which I guess is the problem you have hit:
>>
>> vq->log_guest_addr =
>> translate_log_addr(dev, vq, addr->log_guest_addr);
>> if (vq->log_guest_addr == 0) {
>> RTE_LOG(DEBUG, VHOST_CONFIG,
>> "(%d) failed to map log_guest_addr .\n",
>> dev->vid);
>> return dev;
>> }
>>
>> In the tests I ran I always saw valid log addresses being sent at ring
>> initialization phase, but if, as you claim, it's possible that invalid
>> addresses
>> are given at initialization phase, maybe we should go back to "best-effort"
>> (i.e: remove the return statement)
>>
>> But it's unlikely that qemu has enabled logging at ring initialization so
>> this
>> would effectively disable the translation at the initialization phase. I
>> cannot
>> forecast the consequences of this change without deeper analysis.
>
> That's fine, Adrian. This issue only occurred when using dpdk virtio user
> device.
> Since address logging is disabled in virtio user, simple flag check fix it.
>
What I meant is that we would be changing the behavior for the vhost-user case
in which qemu sends a valid log address even when logging is still not enabled.
In that case we would be effectively postponing the translation.
In any case, I've run a quick test and that case still works fine.
Thanks,
Adrian
>>
>>>> `vq` can be reallocated by numa_realloc.
>>>> We need to update the `addr` pointer before using it.
>>>>
>>>
>>> Hi Tiwei,
>>> Numa_realloc function will copy data from original vq structure to new vq
>> when reallocating.
>>> The content of vhost_ring_addr will be the same in new and old vqs, it
>> may not be necessary to update pointer.
>> That's true but 'addr' still holds a pointer to the old structure, assigned
>> at
>> line 641.
>>
>> Also, note Tiwei's comment regarding updating 'vq->desc'. The idea behind
>> numa_realloc is to reallocate the vhost_virtqueue structure to the same
>> numa
>> node as the descriptor ring. This function is updating the descriptor rings,
>> so
>> I think the idea is to update the ring addresses and then reallocate the
>> virtqueue structure if needed.
>>
> You are right, I misunderstood tiwei's comment. The ring address is useful
> for
> checking numa id, numa reallocation should be after address translation.
>
> Regards,
> Marvin
>
>> Thanks,
>> Adrian
>>
>>> Regards,
>>> Marvin
>>>
>>>> Thanks,
>>>> Tiwei
>>>>
>>>>
>>>>> + vq->log_guest_addr =
>>>>> + translate_log_addr(dev, vq, addr->log_guest_addr);
>>>>> + if (vq->log_guest_addr == 0) {
>>>>> + RTE_LOG(DEBUG, VHOST_CONFIG,
>>>>> + "(%d) failed to map log_guest_addr.\n",
>>>>> + dev->vid);
>>>>> + return dev;
>>>>> + }
>>>>> + }
>>>>> +
>>>>> if (vq_is_packed(dev)) {
>>>>> len = sizeof(struct vring_packed_desc) * vq->size;
>>>>> vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
>>>>> ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
>>>>> - vq->log_guest_addr = 0;
>>>>> if (vq->desc_packed == NULL ||
>>>>> len != sizeof(struct vring_packed_desc) *
>>>>> vq->size) {
>>>>> @@ -655,10 +667,6 @@ translate_ring_addresses(struct virtio_net *dev,
>> int
>>>> vq_index)
>>>>> return dev;
>>>>> }
>>>>>
>>>>> - dev = numa_realloc(dev, vq_index);
>>>>> - vq = dev->virtqueue[vq_index];
>>>>> - addr = &vq->ring_addrs;
>>>>> -
>>>>> len = sizeof(struct vring_packed_desc_event);
>>>>> vq->driver_event = (struct vring_packed_desc_event *)
>>>>> (uintptr_t)ring_addr_to_vva(dev,
>>>>> @@ -701,10 +709,6 @@ translate_ring_addresses(struct virtio_net *dev,
>> int
>>>> vq_index)
>>>>> return dev;
>>>>> }
>>>>>
>>>>> - dev = numa_realloc(dev, vq_index);
>>>>> - vq = dev->virtqueue[vq_index];
>>>>> - addr = &vq->ring_addrs;
>>>>> -
>>>>> len = sizeof(struct vring_avail) + sizeof(uint16_t) * vq->size;
>>>>> if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX))
>>>>> len += sizeof(uint16_t);
>>>>> @@ -741,14 +745,6 @@ translate_ring_addresses(struct virtio_net *dev,
>> int
>>>> vq_index)
>>>>> vq->last_avail_idx = vq->used->idx;
>>>>> }
>>>>>
>>>>> - vq->log_guest_addr =
>>>>> - translate_log_addr(dev, vq, addr->log_guest_addr);
>>>>> - if (vq->log_guest_addr == 0) {
>>>>> - RTE_LOG(DEBUG, VHOST_CONFIG,
>>>>> - "(%d) failed to map log_guest_addr .\n",
>>>>> - dev->vid);
>>>>> - return dev;
>>>>> - }
>>>>> vq->access_ok = 1;
>>>>>
>>>>> VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
>>>>> --
>>>>> 2.17.1
>>>>>
>
