On Mon, Apr 07, 2025 at 12:15:13PM +0200, Morten Brørup wrote: > > From: Bruce Richardson [mailto:bruce.richard...@intel.com] Sent: > > Monday, 7 April 2025 11.49 > > > > On Mon, Apr 07, 2025 at 09:04:05AM +0200, David Marchand wrote: > > > Hello Bruce, > > > > > > On Tue, Apr 1, 2025 at 4:08 PM Bruce Richardson > > > <bruce.richard...@intel.com> wrote: > > > > > > > > On Mon, Mar 24, 2025 at 05:30:26PM +0000, Bruce Richardson wrote: > > > > > Traditionally, DPDK has had a direct mapping of internal lcore- > > ids, to > > > > > the actual core numbers in use. With higher core count servers > > becoming > > > > > more prevalent the issue becomes one of increasing memory > > footprint when > > > > > using such a scheme, due to the need to have all arrays > > dimensioned for > > > > > all cores on the system, whether or not those cores are in use by > > the > > > > > app. > > > > > > > > > > Therefore, the decision was made in the past to not expand the > > > > > build-time RTE_MAX_LCORE value beyond 128. Instead, it was > > recommended > > > > > that users use the "--lcores" EAL parameter to take the high- > > numbered > > > > > cores they wish to use and map them to lcore-ids within the 0 - > > 128 > > > > > range. While this works, this is a little clunky as it means that > > > > > instead of just passing, for example, "-l 130-139", the user must > > > > > instead pass "--lcores 0@130,1@131,2@132,3@133,...." > > > > > > > > > > This patchset attempts to simplify the situation by adding a new > > flag to > > > > > do this mapping automatically. To use cores 130-139 and map them > > to ids > > > > > 0-9 internally, the EAL args now become: "-l 130-139 --map-lcore- > > ids", > > > > > or using the shorter "-M" version of the flag: "-Ml 130-139". > > > > > > > > > > Adding this new parameter required some rework of the existing > > arg > > > > > parsing code, because in current DPDK the args are parsed and > > checked in > > > > > the order they appear on the commandline. This means that using > > the > > > > > example above, the core parameter 130-139 will be rejected > > immediately > > > > > before the "map-lcore-ids" parameter is seen. To work around > > this, the > > > > > core (and service core) parameters are not parsed when seen, > > instead > > > > > they are only saved off and parsed after all arguments are > > parsed. The > > > > > "-l" and "-c" parameters are converted into "--lcores" arguments, > > so all > > > > > assigning of lcore ids is done there in all cases. > > > > > > > > > > RFC->v2: * converted printf to DEBUG log * added "-M" as shorter > > > > > version of flag * added documentation * renamed internal API that > > > > > was changed to avoid any potential > > hidden > > > > > runtime issues. > > > > > > > > > > Bruce Richardson (3): eal: centralize core parameter parsing eal: > > > > > convert core masks and lists to core sets eal: allow automatic > > > > > mapping of high lcore ids > > > > > > > > > Ping for review. > > > > > > > > At a high level, does this feature seem useful to users? > > > > > > This seems useful, though I am not I would touch the existing > > options. > > > I would have gone with a simple -L option (taking the same kind of > > > input than -l but with new behavior), and not combine a flag with > > > existing options. > > > > > > > That would be an easier patchset to do up. However, it would then mean > > that we have no less than 4 different ways to specify the cores to use: > > "- c", "-l", "-L", "--lcores" - and therefore need 4 different sets of > > parsing options for them, and more checks to ensure we have only one of > > the 4 specified in any run. That's why I chose the modifier option, and > > to try and consolidate the core setup a bit. > > > > However, if having a completely new option is preferred, I am happy > > enough to do up a different patchset for that. > > > > > I scanned through the series, not much to say. Maybe add a unit test > > > for new cmdline option. > > > > > Sure. Once it's decided what approach (if any) to take, I'll do up a > > new patchset, taking into account any relevant feedback on this set. > > > > /Bruce > > Changing the EAL parameter parser to a "two pass parser" makes sense. I > think checking for existence of more than one lcore specification options > should suffice; we don't need to accept multiple lcore specification > options and check for conflicts. > > When remapping, do we need to support gaps in the "lcore" (logical cores) > array, e.g. for secondary processes, or can it be continuous from 0 to > the number of specified lcores? > > And are new EAL parameters for this really beneficial? Doesn't e.g. "-l > 0-9@130-139,100@140" suffice? > Actually, I believe "0-9@130-139"[1] is not the same as "0@130,1@131,2@132,...". The latter affinities one thread to one core ten times over, while the former affinitizes 10 threads to 10 cores - leaving those threads free to move about within the 10 cores specified.
Just to confirm, I tweaked our helloworld example to print the cpu affinity of each core when printing. ./build/examples/dpdk-helloworld --no-pci --lcores '(0-3)@(30-33)' EAL: Detected CPU lcores: 96 EAL: Detected NUMA nodes: 2 EAL: Detected static linkage of DPDK EAL: Multi-process socket /run/user/11304126/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: VFIO support initialized hello from core 1, with thread affinity for cores: 30 31 32 33 hello from core 3, with thread affinity for cores: 30 31 32 33 hello from core 2, with thread affinity for cores: 30 31 32 33 hello from core 0, with thread affinity for cores: 30 31 32 33 ./build/examples/dpdk-helloworld --no-pci --lcores '0@30,1@31,2@32,3@33' EAL: Detected CPU lcores: 96 EAL: Detected NUMA nodes: 2 EAL: Detected static linkage of DPDK EAL: Multi-process socket /run/user/11304126/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: VFIO support initialized hello from core 1, with thread affinity for cores: 31 hello from core 3, with thread affinity for cores: hello from core 2, with thread affinity for cores: 32 hello from core 0, with thread affinity for cores: 30 33 Regards, /Bruce [1] This actually needs to be "(0-9)@(130-139)", and with "--lcores", not just "-l", there are actually different flags with different behaviours
