> -----Message d'origine----- > De?: dev [mailto:dev-bounces at dpdk.org] De la part de Thomas Monjalon > Envoy??: vendredi 6 d?cembre 2013 23:24 > ??: Pashupati Kumar > Cc?: dev at dpdk.org > Objet?: Re: [dpdk-dev] Bit spinlocks in DPDK > > 06/12/2013 14:12, Pashupati Kumar : > > From: Thomas Monjalon > > > 06/12/2013 13:04, Pashupati Kumar : > > > > We use bit spinlocks extensively to have compact data structures. > > > > Are there any plans for adding them to DPDK in some future release? > > > > > > Not sure to understand your request. > > > Are you looking for that? > > > http://dpdk.org/doc/api/rte__spinlock_8h.html > > > > I am looking for spinlocks that use a single bit (bit 31) of a 32 bit > > word for locking. The rest of the bits in the word are left > > undisturbed. This enables more compact data structures as only 1 bit > > is consumed for the lock. > > Oh yes, like test_and_set_bit_lock() in Linux: > http://lxr.free- > electrons.com/source/arch/ia64/include/asm/bitops.h?v=3.12#L205 > > I think that a patch would be appreciated :) > > PS: please try to answer below the question. It's far easier to read. > -- > Thomas
Hi, I assume you mean the x64 version, not the ia64: http://lxr.free-electrons.com/source/arch/x86/include/asm/bitops.h. I agree with Pash, the goal of compacting data structures is of the highest importance for high performance networking (HPN). Last week I gave a training on some aspects of multiprocessing, and in particular locking. So I got the mood to check DPDK implementation and I have a few remarks: 1) If the critical section deals with weakly ordered loads then explicit fencing MUST be used: if not, out of order execution will just kill your idea of critical section. Weakly order loads occur on a number of situations involving Write Combining memory (say memory mapped IO). see Intel programming guide 3, 8.1.2.2 Software Controlled Bus Locking and 12.10.3 Streaming Load Hint Instruction. So use rte_mb() or rte_wmb() or rte_rmb() where appropriate. I recommend the rte_unlock code and documentation explains the out of order execution issues and the conditions they have to be mitigated with rte*mb(). I wonder if having an explicit mfence in rte_sinlock_unlock wouldn't be just necessary to avoid "hairy" bugs. In addition, we would have rte_sinlock_unlock_no_mb used internally for performance reasons, and usable externally by advanced users. 2) code that use rte spinlocks are subject to starvation. A simple code with one producer and one consumer demonstrate the issue: I saw either the producer not giving a single chance to a consumer after 1M loops or the consumer grabs the lock right after the first production and loops for ever testing if the producer had produced something (the cycle of unlock/lock is too fast)! Introducing "random" rte_pause() (actually lcore_id number of pauses) solves the problem in a ugly unpredictable manner. It may not create any visible problem on light loads, but may end up being a real issue with several million packets per second. There are a number of scenarios were we don't care if lock algorithm create starvation, but sometimes we do. So I will provide the community with a ticketlock implementation that is starvation free. What is the policy to share a sample program: inline in a mail or as a attachment? What is the policy to submit a patch? Fran?ois-Fr?d?ric
