On 2022-12-18 11:01, Morten Brørup wrote: > This RFC introduces per-thread constructors/destructors: > * Per-thread constructors are functions called as the first thing from newly > created threads. This can be used to initialize variables in thread-local > storage, variables depending on the (tid_t) thread id, and to call setup > functions that must be called from the thread itself. > * Per-thread destructors are functions called (from the thread) as the last > thing when a thread ends. > > At this time, I am seeking feedback on the concept and the proposed > limitations. >
This seems like a useful mechanism, that would simplify things for libraries and PMDs that have dynamically allocated per-thread resources. It's unclear to me, how many such modules there are. > Processes have __attribute__(constructor/destructor) to set up functions to > be called before main(). Nothing similar exists for threads, so we have to > design it ourselves. > > The proposed per-thread constructors/destructors should not apply to all > threads - only to threads created through the DPDK threads API. Agree? > > DPDK has the RTE_INIT()/RTE_FINI() macros for adding process constructors/destructors at build time, so I propose a similar API, i.e. adding RTE_THREAD_INIT() and RTE_THREAD_FINI() macros to build a list of per-thread constructors and destructors at build time. > > Two functions that call the list of per-thread constructors respectively > destructors will be introduced. > > The per-thread constructor function will be called from the newly created > threads within DPDK: > 1. EAL/SERVICE threads: In eal_thread_loop() before the loop. > 2. CTRL threads: In ctrl_thread_init() before start_routine(). > 3. Non-EAL threads: In rte_thread_register(). > > Are any thread types missing in the list above? > Maybe all unregistered non-EAL threads should be excluded, including such created via rte_ctrl_thread_create(). In that case, you would have: 1. EAL threads (both the main and the worker lcores, and both lcores employed as service and non-service lcores). 2. Registered non-EAL threads (created using rte_ctrl_thread_create() or any other means, but having in common having called rte_thread_register()). Another thing which is unclear to me is if libraries/PMDs tend to want to differentiate between EAL threads and registered non-EAL threads. EAL threads are generally required to be safe from preemption and also generally do fast path/packet processing kind of work, while what registered non-EAL threads tend to do (and thus what kind of API access they require) is more open. Another thing to consider is how this mechanism should work in regards to scaling up/down a DPDK application. When an EAL thread is taken out of use, should the finalize function be called? It might complicate things, but on the other hand would allow libraries and PMDs to free resources no longer required. > > The per-thread destructor function will also be registered by the per-thread > constructor, using the POSIX pthread_cleanup_push() function. > > > Examples of where per-thread constructors are useful: > PRNG state initialization [1]: > https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-fc4c610a30810b5d&q=1&e=d8580fcf-a6bb-4e2b-aec4-a772629ece08&u=http%3A%2F%2Finbox.dpdk.org%2Fdev%2F2a5121c7-369f-afde-0898-d45a5b368c3a%40ericsson.com%2F > PMU event setup [2]: > https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-9d4f6edbd54a1ac0&q=1&e=d8580fcf-a6bb-4e2b-aec4-a772629ece08&u=http%3A%2F%2Finbox.dpdk.org%2Fdev%2Fa059c403-8c6c-79c3-c3e9-a8c1815f5a14%40ericsson.com%2FT%2F%23m3d29fbc301059f007425ac148a4113e66d2ef983 > > > Please also refer to the discussion [3] about lcore init callbacks for > further thoughts. Especially the de-facto inability to establish > constructors/destructors at runtime. > > [3]: > https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-94c23fe30fb99087&q=1&e=d8580fcf-a6bb-4e2b-aec4-a772629ece08&u=http%3A%2F%2Finbox.dpdk.org%2Fdev%2F98CBD80474FA8B44BF855DF32C47DC35D875B6%40smartserver.smartshare.dk%2FT%2F%23m23b9a930a4050dc6b0305d3653754bd152c09ab7 > > > Med venlig hilsen / Kind regards, > -Morten Brørup
