Re: [PATCH] generic dynamic per cpu refcounting

2013-01-29 Thread Tejun Heo
Hey, Kent. On Tue, Jan 29, 2013 at 01:45:37PM -0800, Kent Overstreet wrote: > It would definitely be cleaner if the global counter was also 32 bits (I > probably should've done it that way at first) but it works with it being > bigger _provided the sum of the percpu counters is sign extended_ when

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-29 Thread Kent Overstreet
On Tue, Jan 29, 2013 at 12:02:18PM -0800, Tejun Heo wrote: > Hello, Kent. > > On Tue, Jan 29, 2013 at 11:51:41AM -0800, Kent Overstreet wrote: > > > What about overflow? Note that we can have systemetic cases where ref > > > is gotten on one cpu and put on another transferring counts in a > > > s

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-29 Thread Tejun Heo
Hello, Kent. On Tue, Jan 29, 2013 at 11:51:41AM -0800, Kent Overstreet wrote: > > What about overflow? Note that we can have systemetic cases where ref > > is gotten on one cpu and put on another transferring counts in a > > specific direction. > > Heh, this keeps coming up. > > It works becaus

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-29 Thread Kent Overstreet
On Tue, Jan 29, 2013 at 11:29:04AM -0800, Tejun Heo wrote: > Hey, Kent. > > On Tue, Jan 29, 2013 at 08:39:42AM -0800, Kent Overstreet wrote: > > Oh, if this is going to be widely used I should probably have a > > different implementation for archs that don't have atomic64_t in > > hardware. Don't

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-29 Thread Tejun Heo
Hey, Kent. On Tue, Jan 29, 2013 at 08:39:42AM -0800, Kent Overstreet wrote: > Oh, if this is going to be widely used I should probably have a > different implementation for archs that don't have atomic64_t in > hardware. Don't suppose you know the CONFIG_ macro to test against? I > couldn't find i

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-29 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 01:50:42PM -0800, Tejun Heo wrote: > Hello, Kent. > > On Mon, Jan 28, 2013 at 01:45:06PM -0800, Kent Overstreet wrote: > > Ahh. Bias value sounds... hacky (i.e. harder to convince myself it's > > correct) but I see what you're getting at. > > I don't think it's that hacky.

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Tejun Heo
Hello, Kent. On Mon, Jan 28, 2013 at 01:45:06PM -0800, Kent Overstreet wrote: > Ahh. Bias value sounds... hacky (i.e. harder to convince myself it's > correct) but I see what you're getting at. I don't think it's that hacky. Just push the base point to the opposite of zero - LLONG_MAX. The glob

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 01:36:22PM -0800, Tejun Heo wrote: > On Mon, Jan 28, 2013 at 01:28:14PM -0800, Tejun Heo wrote: > > But at that point, the operation is already global, so there gotta be > > a lighter way to synchronize stuff than going through full grace > > period. ie. You can add a bias

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 01:28:14PM -0800, Tejun Heo wrote: > On Mon, Jan 28, 2013 at 01:24:07PM -0800, Kent Overstreet wrote: > > > set dying; > > > synchronize_sched(); > > > collect percpu refs into global atomic_t; > > > put the base ref; > > > > After you set state := dying, percpu_ref

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Tejun Heo
On Mon, Jan 28, 2013 at 01:28:14PM -0800, Tejun Heo wrote: > But at that point, the operation is already global, so there gotta be > a lighter way to synchronize stuff than going through full grace > period. ie. You can add a bias value before marking dead so that the > counter never reaches zero

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Tejun Heo
On Mon, Jan 28, 2013 at 01:24:07PM -0800, Kent Overstreet wrote: > > set dying; > > synchronize_sched(); > > collect percpu refs into global atomic_t; > > put the base ref; > > After you set state := dying, percpu_ref_put() decrements the atomic_t, > but it can't check if it's 0 ye

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 01:18:32PM -0800, Tejun Heo wrote: > Hello, Kent. > > On Mon, Jan 28, 2013 at 12:55:40PM -0800, Kent Overstreet wrote: > > > I don't understand why we need two stages. What prevents the killing > > > thread from fetching percpu counters after dying passes one > > > synchron

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Tejun Heo
Hello, Kent. On Mon, Jan 28, 2013 at 12:55:40PM -0800, Kent Overstreet wrote: > > I don't understand why we need two stages. What prevents the killing > > thread from fetching percpu counters after dying passes one > > synchronize_sched()? > > It does. The second synchronize_sched() is needed aft

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 12:27:45PM -0800, Tejun Heo wrote: > Hey, Kent. > > On Mon, Jan 28, 2013 at 12:22 PM, Kent Overstreet > wrote: > > Could do that too, but then teardown gets really messy for the user - we > > need two synchronize_rcu()s: > > > > state := dying > > > > synchronize_rcu() > >

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Tejun Heo
Hey, Kent. On Mon, Jan 28, 2013 at 12:22 PM, Kent Overstreet wrote: > Could do that too, but then teardown gets really messy for the user - we > need two synchronize_rcu()s: > > state := dying > > synchronize_rcu() > > /* Now nothing's changing the per cpu counters */ > > Add per cpu counters to

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 10:55:52AM -0800, Tejun Heo wrote: > Hey, Kent. > > On Mon, Jan 28, 2013 at 10:49:33AM -0800, Kent Overstreet wrote: > > Yeah. It'd be really nice if it was doable without synchronize_rcu(), > > but it'd definitely make get/put heavier. > > > > Though, re. close() - consid

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Tejun Heo
Hey, Kent. On Mon, Jan 28, 2013 at 10:49:33AM -0800, Kent Overstreet wrote: > Yeah. It'd be really nice if it was doable without synchronize_rcu(), > but it'd definitely make get/put heavier. > > Though, re. close() - considering we only need a synchronize_rcu() if > the ref was in percpu mode, I

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Oleg Nesterov
On 01/28, Kent Overstreet wrote: > > On Fri, Jan 25, 2013 at 07:29:24PM +0100, Oleg Nesterov wrote: > > On 01/25, Oleg Nesterov wrote: > > > > > > > +void __percpu_ref_get(struct percpu_ref *ref, bool alloc) > > > > +{ > > > > + unsigned long pcpu_count; > > > > + uint64_t v; > > > > +

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Mon, Jan 28, 2013 at 10:27:37AM -0800, Tejun Heo wrote: > Hello, guys. > > On Mon, Jan 28, 2013 at 10:15:28AM -0800, Kent Overstreet wrote: > > > percpu_ref_kill(); > > > put_and_dsetroy(); > > > > > > And this can race with another holder which drops the last reference, > > > its put_and_

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Tejun Heo
Hello, guys. On Mon, Jan 28, 2013 at 10:15:28AM -0800, Kent Overstreet wrote: > > percpu_ref_kill(); > > put_and_dsetroy(); > > > > And this can race with another holder which drops the last reference, > > its put_and_dsetroy() can see PCPU_REF_DYING and return false. > > > > Or I misund

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Fri, Jan 25, 2013 at 08:11:39PM +0100, Oleg Nesterov wrote: > On 01/25, Oleg Nesterov wrote: > > > > > +int percpu_ref_kill(struct percpu_ref *ref) > > > +{ > > > ... > > > + if (status == PCPU_REF_PTR) { > > > + unsigned count = 0, cpu; > > > + > > > + synchronize_rcu(); > > > +

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Fri, Jan 25, 2013 at 07:29:24PM +0100, Oleg Nesterov wrote: > On 01/25, Oleg Nesterov wrote: > > > > > +void __percpu_ref_get(struct percpu_ref *ref, bool alloc) > > > +{ > > > + unsigned long pcpu_count; > > > + uint64_t v; > > > + > > > + pcpu_count = ACCESS_ONCE(ref->pcpu_count); > > > + > >

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-28 Thread Kent Overstreet
On Fri, Jan 25, 2013 at 07:09:41PM +0100, Oleg Nesterov wrote: > (add lkml) > > On 01/24, Kent Overstreet wrote: > > > > This has already been on lkml and is in Andrew's tree, Tejun just asked > > me to send it out again: > > I'll try to read this code later, just a couple of questions after a qu

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-25 Thread Oleg Nesterov
On 01/25, Oleg Nesterov wrote: > > > +int percpu_ref_kill(struct percpu_ref *ref) > > +{ > > ... > > + if (status == PCPU_REF_PTR) { > > + unsigned count = 0, cpu; > > + > > + synchronize_rcu(); > > + > > + for_each_possible_cpu(cpu) > > + count +=

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-25 Thread Oleg Nesterov
On 01/25, Oleg Nesterov wrote: > > > +void __percpu_ref_get(struct percpu_ref *ref, bool alloc) > > +{ > > + unsigned long pcpu_count; > > + uint64_t v; > > + > > + pcpu_count = ACCESS_ONCE(ref->pcpu_count); > > + > > + if (REF_STATUS(pcpu_count) == PCPU_REF_PTR) { > > + /* for rc

Re: [PATCH] generic dynamic per cpu refcounting

2013-01-25 Thread Oleg Nesterov
(add lkml) On 01/24, Kent Overstreet wrote: > > This has already been on lkml and is in Andrew's tree, Tejun just asked > me to send it out again: I'll try to read this code later, just a couple of questions after a quick glance. Sorry this was already discussed... > +struct percpu_ref { > +