> From: Jerin Jacob [mailto:jerin.ja...@caviumnetworks.com] > Sent: Friday, June 30, 2017 5:45 AM > To: Richardson, Bruce <bruce.richard...@intel.com> > Cc: Van Haaren, Harry <harry.van.haa...@intel.com>; dev@dpdk.org; > tho...@monjalon.net; > Wiles, Keith <keith.wi...@intel.com> > Subject: Re: Service lcores and Application lcores > > -----Original Message----- > > Date: Thu, 29 Jun 2017 16:57:08 +0100 > > From: Bruce Richardson <bruce.richard...@intel.com> > > To: "Van Haaren, Harry" <harry.van.haa...@intel.com> > > CC: "dev@dpdk.org" <dev@dpdk.org>, 'Jerin Jacob' > > <jerin.ja...@caviumnetworks.com>, "tho...@monjalon.net" > > <tho...@monjalon.net>, "Wiles, Keith" <keith.wi...@intel.com> > > Subject: Re: Service lcores and Application lcores > > User-Agent: Mutt/1.8.1 (2017-04-11) > > > > On Thu, Jun 29, 2017 at 03:36:04PM +0100, Van Haaren, Harry wrote: > > > Hi All,
<snip> > > > A proposal for Eventdev, to ensure Service lcores and Application lcores > > > play nice; > > > > > > 1) Application lcores must not directly call rte_eventdev_schedule() > > > 2A) Service cores are the proper method to run services > > > 2B) If an application insists on running a service "manually" on an app > > > lcore, we > provide a function for that: > > > rte_service_run_from_app_lcore(struct service *srv); > > > > > > The above function would allow a pesky app to run services on its own > > > (non-service > core) lcores, but > > > does so through the service-core framework, allowing the service-library > > > atomic to > keep access serialized as required for non-multi-thread-safe services. > > > > > > The above solution maintains the option of running the eventdev PMD as > > > now (single- > core dedicated to a single service), while providing correct serialization by > using the > rte_service_run_from_app_lcore() function. Given the atomic is only used when > required > (multiple cores mapped to the service) there should be no performance delta. > > > > > > Given that the application should not invoke rte_eventdev_schedule(), we > > > could even > consider removing it from the Eventdev API. A PMD that requires cycles > registers a > service, and an application can use a service core or the > run_from_app_lcore() function if > it wishes to invoke that service on an application owned lcore. > > > > > > > > > Opinions? > > > > I would be in favour of this proposal, except for the proposed name for > > the new function. It would be useful for an app to be able to "adopt" a > > service into it's main loop if so desired. If we do this, I think I'd > > +1 > > Agree with Harry and Bruce here. > > I think, The adapter function should take "struct service *" and return > lcore_function_t so that it can run using exiting rte_eal_remote_launch() I don't think providing a remote-launch API is actually beneficial. Remote-launching a single service is equivalent to adding that lcore as a service-core, and mapping it to just that single service. The advantage of adding it as a service core, is future-proofing for if more services need to be added to that core in future, and statistics of the service core infrastructure. A convenience API could be provided to perform the core_add(), service_start(), enable_on_service() and core_start() APIs in one. Also, the remote_launch API doesn't solve the original problem - what if an application lcore wishes to run one iteration of a service "manually". The remote_launch style API does not solve this problem. Here a much simpler API to run a service... as a counter-proposal :) /** Runs one iteration of *service* on the calling lcore */ int rte_service_iterate(struct rte_service_spec *service); The iterate() function can check that the service is start()-ed, check the number of mapped-lcores and utilize the atomic to prevent concurrent access to multi-thread unsafe services. By exposing the function-pointer/userdata directly, we lose that. Thinking about it, a function like rte_service_iterate() is the only functionally correct approach. (Exposing the callback directly brings us back to the "application thread without atomic check" problem.) Thoughts? > > also support the removal of a dedicated schedule call from the eventdev > > API, or alternatively, if it is needed by other PMDs, leave it as a > > no-op in the sw PMD in favour of the service-cores managed function. > > I would be in favor of removing eventdev schedule and > RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability so that it is completely > transparent to application whether scheduler runs on HW or SW or "combination > of both" Yep this bit sounds good!
