Re: [dpdk-dev] [PATCH] event/octeontx2: use wfe while waiting for head

[Gavin Hu (Arm Technology China)]

Hi Pavan,

> -----Original Message-----
> From: pbhagavat...@marvell.com <pbhagavat...@marvell.com>
> Sent: Thursday, October 24, 2019 12:13 AM
> To: Gavin Hu (Arm Technology China) <gavin...@arm.com>;
> jer...@marvell.com; Pavan Nikhilesh <pbhagavat...@marvell.com>
> Cc: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH] event/octeontx2: use wfe while waiting for
> head
> 
> From: Pavan Nikhilesh <pbhagavat...@marvell.com>
> 
> Use wfe to save power while waiting for tag to become head.
> 
> SSO signals EVENTI to allow cores to exit from wfe when they
> are waiting for specific operations in which one of them is
> setting HEAD bit in GWS_TAG.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavat...@marvell.com>
> ---
>  drivers/event/octeontx2/otx2_worker.h | 30 ++++++++++++++++++++++++--
> -
>  1 file changed, 27 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/event/octeontx2/otx2_worker.h
> b/drivers/event/octeontx2/otx2_worker.h
> index 4e971f27c..7a55caca5 100644
> --- a/drivers/event/octeontx2/otx2_worker.h
> +++ b/drivers/event/octeontx2/otx2_worker.h
> @@ -226,10 +226,34 @@ otx2_ssogws_swtag_wait(struct otx2_ssogws *ws)
>  }
> 
>  static __rte_always_inline void
> -otx2_ssogws_head_wait(struct otx2_ssogws *ws, const uint8_t wait_flag)
> +otx2_ssogws_head_wait(struct otx2_ssogws *ws)
>  {
> -     while (wait_flag && !(otx2_read64(ws->tag_op) & BIT_ULL(35)))
> +#ifdef RTE_ARCH_ARM64
> +     uint64_t tag;
> +
> +     asm volatile (
> +                     "       ldr %[tag], [%[tag_op]]         \n"
"ldxr" should be used, exclusive-load is required to "monitor" the location, 
then a write to the location will cause clear of the exclusive monitor, thus a 
wake up event is generated implicitly.
You can find more explanation is here:
http://inbox.dpdk.org/dev/am0pr08mb5363f9d1ba158b66b803ea068f...@am0pr08mb5363.eurprd08.prod.outlook.com/
 
/Gavin
> +                     "       tbnz %[tag], 35, done%=         \n"
> +                     "       sevl                            \n"
> +                     "rty%=: wfe                             \n"
> +                     "       ldr %[tag], [%[tag_op]]         \n"
> +                     "       tbz %[tag], 35, rty%=           \n"
> +                     "done%=:                                \n"
> +                     : [tag] "=&r" (tag)
> +                     : [tag_op] "r" (ws->tag_op)
> +                     );
> +#else
> +     /* Wait for the HEAD to be set */
> +     while (!(otx2_read64(ws->tag_op) & BIT_ULL(35)))
>               ;
> +#endif
> +}
> +
> +static __rte_always_inline void
> +otx2_ssogws_order(struct otx2_ssogws *ws, const uint8_t wait_flag)
> +{
> +     if (wait_flag)
> +             otx2_ssogws_head_wait(ws);
> 
>       rte_cio_wmb();
What ordering does this barrier try to keep?  If there is a write then wait for 
kind of response, should this barrier move before  otx2_ssogws_head_wait?
/Gavin
>  }
> @@ -258,7 +282,7 @@ otx2_ssogws_event_tx(struct otx2_ssogws *ws,
> struct rte_event ev[],
> 
>       /* Perform header writes before barrier for TSO */
>       otx2_nix_xmit_prepare_tso(m, flags);
> -     otx2_ssogws_head_wait(ws, !ev->sched_type);
> +     otx2_ssogws_order(ws, !ev->sched_type);
>       otx2_ssogws_prepare_pkt(txq, m, cmd, flags);
> 
>       if (flags & NIX_TX_MULTI_SEG_F) {
> --
> 2.17.1