On Sun, Nov 15, 2015 at 09:29:59PM +0200, Michael S. Tsirkin wrote:
> Clarify logging setup to make sure all clients comply in a way that is
> future-proof. Document how rings are started/stopped.
>
> Signed-off-by: Michael S. Tsirkin <m...@redhat.com>
> ---
> docs/specs/vhost-user.txt | 62
> ++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 53 insertions(+), 9 deletions(-)
>
> diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
> index 26dde2e..6a0ed8f 100644
> --- a/docs/specs/vhost-user.txt
> +++ b/docs/specs/vhost-user.txt
> @@ -87,6 +87,14 @@ Depending on the request type, payload can be:
> User address: a 64-bit user address
> mmap offset: 64-bit offset where region starts in the mapped memory
>
> +* Log description
> + ---------------------------
> + | log size | log offset |
> + ---------------------------
> + log size: size of area used for logging
> + log offset: offset from start of supplied file descriptor
> + where logging starts (i.e. where guest address 0 would be logged)
> +
> In QEMU the vhost-user message is implemented with the following struct:
>
> typedef struct VhostUserMsg {
> @@ -138,6 +146,23 @@ As older slaves don't support negotiating protocol
> features,
> a feature bit was dedicated for this purpose:
> #define VHOST_USER_F_PROTOCOL_FEATURES 30
>
> +Starting and stopping rings
> +----------------------
> +Each ring is initialized in a stopped state, client must not process it until
> +ring is enabled.
> +
> +If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, client must start and
> +stop ring processing upon receiving VHOST_USER_SET_VRING_ENABLE with
> parameters
> +1 and 0 respoectively.
> +
> +If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, client must start
> +ring processing upon receiving a kick (that is, detecting that file
> descriptor
> +is readable) on the descriptor specified by VHOST_USER_SET_VRING_KICK, and
> stop
> +ring processing upon receiving VHOST_USER_GET_VRING_BASE.
> +
> +While rings are running, client must support changing some configuration
> +aspects on the fly.
> +
> Multiple queue support
> ----------------------
>
> @@ -162,9 +187,13 @@ the slave makes to the memory mapped regions. The client
> should mark
> the dirty pages in a log. Once it complies to this logging, it may
> declare the VHOST_F_LOG_ALL vhost feature.
>
> +To start/stop logging of data/used ring writes, server may send messages
> +VHOST_USER_SET_FEATURES with VHOST_F_LOG_ALL and VHOST_USER_SET_VRING_ADDR
> with
> +VHOST_VRING_F_LOG in ring's flags set to 1/0, respectively.
> +
> All the modifications to memory pointed by vring "descriptor" should
> be marked. Modifications to "used" vring should be marked if
> -VHOST_VRING_F_LOG is part of ring's features.
> +VHOST_VRING_F_LOG is part of ring's flags.
>
> Dirty pages are of size:
> #define VHOST_LOG_PAGE 0x1000
> @@ -173,22 +202,35 @@ The log memory fd is provided in the ancillary data of
> VHOST_USER_SET_LOG_BASE message when the slave has
> VHOST_USER_PROTOCOL_F_LOG_SHMFD protocol feature.
>
> -The size of the log may be computed by using all the known guest
> -addresses. The log covers from address 0 to the maximum of guest
> +The size of the log is supplied as part of VhostUserMsg
> +which should be large enough to cover all known guest
> +addresses. Log starts at the supplied offset in the
> +supplied file descriptor.
> +The log covers from address 0 to the maximum of guest
> regions. In pseudo-code, to mark page at "addr" as dirty:
>
> page = addr / VHOST_LOG_PAGE
> log[page / 8] |= 1 << page % 8
>
> +Where addr is the guest physical address.
> +
> Use atomic operations, as the log may be concurrently manipulated.
>
> +Note that when logging modifications to the used ring (when VHOST_VRING_F_LOG
> +is set for this ring), log_guest_addr should be used to calculate the log
> +offset: the write to first byte of the used ring is logged at this offset
> from
> +log start. Also note that this value might be outside the legal guest
> physical
> +address range (i.e. does not have to be covered by the VhostUserMemory
> table),
> +but the bit offset of the last byte of the ring must fall within
> +the size supplied by VhostUserLog.
> +
> VHOST_USER_SET_LOG_FD is an optional message with an eventfd in
> ancillary data, it may be used to inform the master that the log has
> been modified.
>
> -Once the source has finished migration, VHOST_USER_RESET_OWNER message
> -will be sent by the source. No further update must be done before the
> -destination takes over with new regions & rings.
> +Once the source has finished migration, rings will be stopped by
> +the source. No further update must be done before rings are
> +restarted.
>
> Protocol features
> -----------------
> @@ -259,11 +301,11 @@ Message types
> * VHOST_USER_RESET_OWNER
>
> Id: 4
> - Equivalent ioctl: VHOST_RESET_OWNER
> Master payload: N/A
>
> - Issued when a new connection is about to be closed. The Master will no
> - longer own this connection (and will usually close it).
> + This is no longer used. Used to be sent to request stopping
> + all rings, but some clients interpreted it to also discard
> + connection state (this interpretation would lead to bugs).
The new code uses VHOST_USER_RESET_DEVICE as the name for request
Id 4. Also it is not clear whether it is recommended just to
ignore the reset device request now?
>
> * VHOST_USER_SET_MEM_TABLE
>
> @@ -388,6 +430,8 @@ Message types
> Master payload: vring state description
>
> Signal slave to enable or disable corresponding vring.
> + This request should be sent only when VHOST_USER_F_PROTOCOL_FEATURES
> + has been negotiated.
>
> * VHOST_USER_SEND_RARP
>
> --
> MST
--Victor
