Thanks for the patch.
Is there any reason why core 0 is treated specially?
I think we should put pmd_thread_setpriority in lib/ovs-numa.c (adding
a ovs_numa prefix), and do nothing if dummy_numa is false. Or perhaps
integrate it the pthread_setschedparam in
ovs_numa_thread_setaffinity_core().
I've noticed that processes with the same affinity as a PMD thread will
become
totally unresponsive after this patch. Is this expected? Will this have a
negative
impact on the overall stability of the system?
2016-07-05 13:05 GMT-07:00 Bhanuprakash Bodireddy <
bhanuprakash.bodire...@intel.com>:
> Set the DPDK pmd thread scheduling policy to SCHED_RR and static
> priority to highest priority value of the policy. This is to deal with
> pmd thread starvation case where another cpu hogging process can get
> scheduled/affinitized on to the same core the pmd thread is running there
> by significantly impacting the datapath performance.
>
> Setting the realtime scheduling policy to the pmd threads is one step
> towards Fastpath Service Assurance in OVS DPDK.
>
> The realtime scheduling policy is applied only when CPU mask is passed
> to 'pmd-cpu-mask'. The exception to this is 'pmd-cpu-mask=1', where the
> policy and priority shall not be applied to pmd thread spawned on core0.
> For example:
>
> * In the absence of pmd-cpu-mask or if pmd-cpu-mask=1, one pmd
> thread shall be created and affinitized to 'core 0' with default
> scheduling policy and priority applied.
>
> * If pmd-cpu-mask is specified with CPU mask > 1, one or more pmd
> threads shall be spawned on the corresponding core(s) in the mask
> and real time scheduling policy SCHED_RR and highest static
> priority is applied to the pmd thread(s).
>
> To reproduce the issue use following commands:
>
> ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=6
> taskset 0x2 cat /dev/zero > /dev/null &
>
> Also OVS control threads should not be affinitized to the pmd cores.
> For example 'dpdk-lcore-mask' and 'pmd-cpu-mask' should be exclusive.
>
> v1->v2:
> * Removed #ifdef and introduced dummy function "pmd_thread_setpriority"
> in netdev-dpdk.h
> * Rebase
>
> Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodire...@intel.com>
> ---
> lib/dpif-netdev.c | 8 ++++++++
> lib/netdev-dpdk.c | 14 ++++++++++++++
> lib/netdev-dpdk.h | 7 +++++++
> 3 files changed, 29 insertions(+)
>
> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> index 37c2631..6ff81d6 100644
> --- a/lib/dpif-netdev.c
> +++ b/lib/dpif-netdev.c
> @@ -2849,6 +2849,14 @@ pmd_thread_main(void *f_)
> ovs_numa_thread_setaffinity_core(pmd->core_id);
> dpdk_set_lcore_id(pmd->core_id);
> poll_cnt = pmd_load_queues_and_ports(pmd, &poll_list);
> +
> + /* Set pmd thread's scheduling policy to SCHED_RR and priority to
> + * highest priority of SCHED_RR policy, In absence of pmd-cpu-mask
> (or)
> + * pmd-cpu-mask=1, default scheduling policy and priority shall
> + * apply to pmd thread */
> + if (pmd->core_id) {
> + pmd_thread_setpriority(SCHED_RR);
> + }
> reload:
> emc_cache_init(&pmd->flow_cache);
>
> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
> index 02e2c58..ce1683b 100644
> --- a/lib/netdev-dpdk.c
> +++ b/lib/netdev-dpdk.c
> @@ -3541,3 +3541,17 @@ dpdk_thread_is_pmd(void)
> {
> return rte_lcore_id() != NON_PMD_CORE_ID;
> }
> +
> +void
> +pmd_thread_setpriority(int policy)
> +{
> + struct sched_param threadparam;
> + int err;
> +
> + memset(&threadparam, 0, sizeof(threadparam));
> + threadparam.sched_priority = sched_get_priority_max(policy);
> + err = pthread_setschedparam(pthread_self(), policy, &threadparam);
> + if (err) {
> + VLOG_WARN("Thread priority error %d",err);
> + }
> +}
> diff --git a/lib/netdev-dpdk.h b/lib/netdev-dpdk.h
> index 80bb834..1890ae4 100644
> --- a/lib/netdev-dpdk.h
> +++ b/lib/netdev-dpdk.h
> @@ -26,6 +26,7 @@ struct smap;
> void netdev_dpdk_register(void);
> void free_dpdk_buf(struct dp_packet *);
> void dpdk_set_lcore_id(unsigned cpu);
> +void pmd_thread_setpriority(int policy);
>
> #else
>
> @@ -51,6 +52,12 @@ dpdk_set_lcore_id(unsigned cpu OVS_UNUSED)
> /* Nothing */
> }
>
> +static inline void
> +pmd_thread_setpriority(int policy OVS_UNUSED)
> +{
> + /* Nothing */
> +}
> +
> #endif /* DPDK_NETDEV */
>
> void dpdk_init(const struct smap *ovs_other_config);
> --
> 2.4.11
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
>
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
