On Wed 23 Oct 2019 at 15:49, Jamal Hadi Salim <j...@mojatatu.com> wrote: > Hi Vlad, > > On 2019-10-22 10:17 a.m., Vlad Buslov wrote: >> Currently, significant fraction of CPU time during TC filter allocation >> is spent in percpu allocator. Moreover, percpu allocator is protected >> with single global mutex which negates any potential to improve its >> performance by means of recent developments in TC filter update API that >> removed rtnl lock for some Qdiscs and classifiers. In order to >> significantly improve filter update rate and reduce memory usage we >> would like to allow users to skip percpu counters allocation for >> specific action if they don't expect high traffic rate hitting the >> action, which is a reasonable expectation for hardware-offloaded setup. >> In that case any potential gains to software fast-path performance >> gained by usage of percpu-allocated counters compared to regular integer >> counters protected by spinlock are not important, but amount of >> additional CPU and memory consumed by them is significant. > > Great to see this becoming low hanging on the fruit tree > after your improvements. > Note: had a discussion a few years back with Eric D.(on Cc) > when i was trying to improve action dumping; what you are seeing > was very visible when doing a large batch creation of actions. > At the time i was thinking of amortizing the cost of that mutex > in a batch action create i.e you ask the per cpu allocator > to alloc a batch of the stats instead of singular. > > I understand your use case being different since it is for h/w > offload. If you have time can you test with batching many actions > and seeing the before/after improvement?
Will do. > > Note: even for h/w offload it makes sense to first create the actions > then bind to filters (in my world thats what we end up doing). > If we can improve the first phase it is a win for both s/w and hw use > cases. > > Question: > Given TCA_ACT_FLAGS_FAST_INIT is common to all actions would it make > sense to use Could you have used a TLV in the namespace of TCA_ACT_MAX > (outer TLV)? You will have to pass a param to ->init(). It is not common for all actions. I omitted modifying actions that are not offloaded and some actions don't user percpu allocator at all (pedit, for example) and have no use for this flag at the moment. > > cheers, > jamal
