On (01/24/24 16:53), David Marchand wrote: > Date: Wed, 24 Jan 2024 16:53:33 +0100 > From: David Marchand <david.march...@redhat.com> > To: Rahul Gupta <rahulg...@linux.microsoft.com> > Cc: dev@dpdk.org, tho...@monjalon.net, bruce.richard...@intel.com, > dmitry.kozl...@gmail.com, step...@networkplumber.org, > sovar...@linux.microsoft.com, ok...@kernel.org, > sujithsan...@microsoft.com, sowmini.varad...@microsoft.com, > krathina...@microsoft.com, rahulrgupt...@gmail.com, Rahul Gupta > <rahulg...@microsoft.com> > Subject: Re: [dpdk-dev] [PATCH v4] eal: refactor rte_eal_init into > sub-functions > > On Wed, Jan 24, 2024 at 2:45 PM Rahul Gupta > <rahulg...@linux.microsoft.com> wrote: > > > > From: Rahul Gupta <rahulg...@microsoft.com> > > > > In continuation to the following email, I am sending this patch. > > (https://inbox.dpdk.org/dev/20231110172523.ga17...@microsoft.com/) > > > > Initialization requires rte_eal_init + rte_pktmbuf_pool_create which > > can consume a total time of 500-600 ms: > > a) For many devices FLR may take a significant chunk of time > > (200-250 ms in our use-case), this FLR is triggered during device > > probe in rte_eal_init(). > > b) rte_pktmbuf_pool_create() can consume up to 300-350 ms for > > applications that require huge memory. > > > > This cost is incurred on each restart (which happens in our use-case > > during binary updates for servicing). > > This patch provides an optimization using pthreads that applications > > can use and which can save 200-230ms. > > > > In this patch, rte_eal_init() is refactored into two parts- > > a) 1st part is dependent code ie- it’s a perquisite of the FLR and > > mempool creation. So this code needs to be executed before any > > pthreads. Its named as rte_eal_init_setup() > > b) 2nd part of code is independent code ie- it can execute in parallel > > to mempool creation in a pthread. Its named as > > rte_eal_init_async_setup(). > > > > In existing applications no changes are required unless they wish to > > leverage > > the optimization. > > > > If the application wants to leverage this optimization, then it needs to > > call > > rte_eal_init_async() (instead of call rte_eal_init()), then it can create a > > thread using rte_eal_remote_launch() to schedule a task it would like todo > > in > > parallel rte_eal_init_async_setup(), this task can be a mbuf pool creation > > using- rte_pktmbuf_pool_create() > > > > After this, if next operations require completion of above task, then > > user can use rte_eal_init_wait_async_setup_complete(), > > or if user wants to just check status of that thread, then use- > > rte_eal_init_async_setup_done() > > Looking at what this patch does.. I am under the impression all you > really need is rte_eal_init without initial probing. > Such behavior can probably be achieved with a allowlist set to a non > existing device (like for example "-a 0000:00:00.0"), then later, use > device hotplug. The patch will be useful to all the adapters irrespective of their host plug support. > > Some quick note on this patch. > > - don't expose symbols externally if they are only for internal use in > the same library, done in next patch. > - current version is 24.03, not 24.01 (wrt comment in version.map), done > - other OSes are not handled by this patch, please do the work for > FreeBSD and Windows, I can send patch to support non-linux OS, but due to lack of setup, I will need help to test same.
Also, I am planning to do the porting at the end (ie after incorporating all review comments, in order to prevent duplication of efforts). > - as a followup of the previous point, please check if we can share > code between OSes and, if so, move the shared code to lib/eal/common, The code for rte_eal_init() is different for all three distros, (even if I consider just the 1st part of rte_eal_init() ie rte_eal_init_setup()). So its difficult to move to common dir. We will have todo it separately for all OS. > - did you test this series with multiprocess? Yes it works fine, I have tested it with simple_mp app. > - why should telemetry and other parts of the current rte_eal_init() > be left in the second stage of this initialisation pipeline? Actually motivation of this patch was todo most of the work in parallel ie in second stage, so not only probe/FLR but telemetry and any other work which can be executed in parallel are done here. (pls refer to the link in the commit message for more details) > > > -- > David Marchand Thanks for the reviewing, my comments are inline. Thanks, Rahul.
