<lots of snips to make responses consumable!>
> From: Jerin Jacob [mailto:jerin.ja...@caviumnetworks.com]
> To: Van Haaren, Harry <harry.van.haa...@intel.com>
> Cc: dev@dpdk.org; tho...@monjalon.net; Wiles, Keith <keith.wi...@intel.com>;
> Richardson,
> Bruce <bruce.richard...@intel.com>
> Subject: Re: [PATCH v4 1/7] service cores: header and implementation
>
<snip>
>
> Remove above info from the git commit.
Done
> Fix the below mentioned documentation warning.
>
> +/export/dpdk.org/lib/librte_eal/common/include/rte_service.h:338:
> warning: argument 'enabled' of command @param is not found in the
> argument list of rte_service_set_stats_enable(int enable)
> +/export/dpdk.org/lib/librte_eal/common/include/rte_service.h:346:
> warning: The following parameters of rte_service_set_stats_enable(int
> enable) are not documented:
> + parameter 'enable'
> +/export/dpdk.org/lib/librte_eal/common/include/rte_service.h:349:
> warning: argument 'The' of command @param is not found in the argument
> list of rte_service_lcore_list(uint32_t array[], uint32_t n)
> +/export/dpdk.org/lib/librte_eal/common/include/rte_service.h:367:
> warning: The following parameters of rte_service_lcore_list(uint32_t
> array[], uint32_t n) are not documented:
> + parameter 'n'
Done
> command to reproduce:
> ./devtools/test-build.sh -j8 x86_64-native-linuxapp-gcc+shared
> x86_64-native-linuxapp-
> gcc+debug
Thanks - noted.
> > +
> > +/* the internal values of a service core */
> > +struct core_state {
>
> Change to lcore_state.
Done.
> > +
> > +void rte_service_set_stats_enable(int enabled)
>
> IMO, It should be per service i.e
> rte_service_set_stats_enable(const struct rte_service_spec *spec, int enable)
Improved service library to handle statistics collection on a per service basis.
> > + /* check if this is the only core mapped, else use
> > + * atomic to serialize cores mapped to this service
> > + */
> > + uint32_t *lock = (uint32_t *)&s->execute_lock;
> > + if ((s->spec.capabilities & RTE_SERVICE_CAP_MT_SAFE) ||
> > + (s->num_mapped_cores == 1 ||
> > + rte_atomic32_cmpset(lock, 0, 1))) {
> > + void *userdata = s->spec.callback_userdata;
> > +
> > + if (cs->collect_statistics) {
> > + uint64_t start = rte_rdtsc();
> > + s->spec.callback(userdata);
> > + uint64_t end = rte_rdtsc();
> > + s->cycles_spent += end - start;
> > + cs->calls_per_service[i]++;
> > + s->calls++;
> > + } else
> > + s->spec.callback(userdata);
> > +
> > + if ((s->spec.capabilities &
> > + RTE_SERVICE_CAP_MT_SAFE) == 0 &&
> > + s->num_mapped_cores > 1)
>
> How about computing the non rte_atomic32_cmpset() mode value first and
> using in both place i.e here and in the top "if" loop
>
> const int need_cmpset = (s->spec.capabilities &
> RTE_SERVICE_CAP_MT_SAFE)...
> if (need_cmpset || rte_atomic32_cmpset(lock, 0, 1))
> ..
> if (need_cmpset)
> rte_atomic32_clear()..
Yes good idea. Indeed I wasn't happy with that, this is a good fix.
The checks to detect if we need_cmpset are a little complex, but it's the only
solution I see.
I've added another unit test to verify both MT safe and MT unsafe callback
operations.
> > +int32_t
> > +rte_service_set_default_mapping(void)
> > +{
> > + /* create a default mapping from cores to services, then start the
> > + * services to make them transparent to unaware applications.
> > + */
> > + uint32_t i;
> > + int ret;
> > + uint32_t count = rte_service_get_count();
> > +
> > + int32_t lcore_iter = 0;
> > + uint32_t ids[RTE_MAX_LCORE];
> > + int32_t lcore_count = rte_service_lcore_list(ids, RTE_MAX_LCORE);
> > +
> > + for (i = 0; i < count; i++) {
> > + struct rte_service_spec *s = rte_service_get_by_id(i);
> > + if (!s)
> > + return -EINVAL;
> > +
> > + /* if no lcores available as services cores, don't setup map.
> > + * This means app logic must add cores, and setup mappings
> > + */
> > + if (lcore_count > 0) {
>
>
> > + /* do 1:1 core mapping here, with each service getting
> > + * assigned a single core by default. Adding multiple
> > + * services should multiplex to a single core, or 1:1
> > + * if services == cores
> > + */
> > + ret = rte_service_enable_on_lcore(s, ids[lcore_iter]);
> > + if (ret)
> > + return -ENODEV;
> > + }
> > +
> > + lcore_iter++;
> > + if (lcore_iter >= lcore_count)
> > + lcore_iter = 0;
> > +
> > + ret = rte_service_start(s);
>
> IMO, we don't need to start the service if lcore_count == 0. How about
> moving the "if (lcore_count > 0)" check on top of for the loop and exist
> from the function if lcore_count == 0.
Good point, added if() check for lcore_count at start of function, and return
if no service cores are available.
> > +
> > +int32_t
> > +rte_service_lcore_add(uint32_t lcore)
> > +{
> > + if (lcore >= RTE_MAX_LCORE)
> > + return -EINVAL;
> > + if (cores_state[lcore].is_service_core)
> > + return -EALREADY;
> > +
> > + set_lcore_state(lcore, ROLE_SERVICE);
> > +
> > + /* ensure that after adding a core the mask and state are defaults */
> > + cores_state[lcore].service_mask = 0;
> > + cores_state[lcore].runstate = RUNSTATE_STOPPED;
>
> If worker core can call rte_service_lcore_add() then add rte_smp_wmb()
> here. Applies to rte_service_lcore_del() as well.
Added barriers to both add() and del().
