> -----Original Message-----
> From: Jakub Kicinski <k...@kernel.org>
> Sent: Thursday, August 15, 2024 12:09 PM
> To: Erni Sri Satya Vennela <er...@linux.microsoft.com>
> Cc: KY Srinivasan <k...@microsoft.com>; Haiyang Zhang
> <haiya...@microsoft.com>; wei....@kernel.org; Dexuan Cui
> <de...@microsoft.com>; da...@davemloft.net; eduma...@google.com;
> pab...@redhat.com; linux-hyperv@vger.kernel.org; net...@vger.kernel.org;
> linux-ker...@vger.kernel.org; Erni Sri Satya Vennela <er...@microsoft.com>
> Subject: Re: [PATCH v2] net: netvsc: Update default VMBus channels
>
> On Wed, 14 Aug 2024 09:59:13 -0700 Erni Sri Satya Vennela wrote:
> > Change VMBus channels macro (VRSS_CHANNEL_DEFAULT) in
> > Linux netvsc from 8 to 16 to align with Azure Windows VM
> > and improve networking throughput.
> >
> > For VMs having less than 16 vCPUS, the channels depend
> > on number of vCPUs. Between 16 to 32 vCPUs, the channels
> > default to VRSS_CHANNEL_DEFAULT. For greater than 32 vCPUs,
> > set the channels to number of physical cores / 2 as a way
> > to optimize CPU resource utilization and scale for high-end
> > processors with many cores.
> > Maximum number of channels are by default set to 64.
> >
> > Based on this change the subchannel creation would change as follows:
> >
> > -------------------------------------------------------------
> > |No. of vCPU |dev_info->num_chn |subchannel created |
> > -------------------------------------------------------------
> > | 0-16 | 16 | vCPU |
> > | >16 & <=32 | 16 | 16 |
> > | >32 & <=128 | vCPU/2 | vCPU/2 |
> > | >128 | vCPU/2 | 64 |
> > -------------------------------------------------------------
> >
> > Performance tests showed significant improvement in throughput:
> > - 0.54% for 16 vCPUs
> > - 0.83% for 32 vCPUs
> > - 1.76% for 48 vCPUs
> > - 10.35% for 64 vCPUs
> > - 13.47% for 96 vCPUs
>
> Is there anything that needs clarifying in my feedback on v1?
>
> https://lore.kernel.org/all/20240807201857.445f9...@kernel.org/
>
> Ignoring maintainer feedback is known to result in angry outbursts.
Your suggestion on netif_get_num_default_rss_queues() is not ignored.
We discussed internally on the formula we used for the num_chn, and
chose a similar formula for higher number of vCPUs as in
netif_get_num_default_rss_queues().
For lower number of vCPUs, we use the same default as Windows guests,
because we don't want any potential regression.
So, the end result is a step function:
> > -------------------------------------------------------------
> > |No. of vCPU |dev_info->num_chn |subchannel created |
> > -------------------------------------------------------------
> > | 0-16 | 16 | vCPU |
> > | >16 & <=32 | 16 | 16 |
> > | >32 & <=128 | vCPU/2 | vCPU/2 |
> > | >128 | vCPU/2 | 64 |
> > -------------------------------------------------------------
@Erni Sri Satya Vennela
Next time, please reply to maintainer's emails to LKML, regarding
how you think of the suggestions.
Also, netif_get_num_default_rss_queues() uses #phys cores, which
is different from num_online_cpus().
You can try like below, in addition to your comparison test, see
if it's better than the patch v2.
dev_info->num_chn = netif_get_num_default_rss_queues();
if (dev_info->num_chn < VRSS_CHANNEL_DEFAULT)
dev_info->num_chn = VRSS_CHANNEL_DEFAULT;
Thanks,
- Haiyang
