Re: pagecache locking

2019-07-10 Thread Jan Kara
On Wed 10-07-19 09:47:12, Dave Chinner wrote: > On Mon, Jul 08, 2019 at 03:31:14PM +0200, Jan Kara wrote: > > I'd be really careful with nesting range locks. You can have nasty > > situations like: > > > > Thread 1Thread 2 > > read_lock(0,1000) > > write_lock(500

Re: pagecache locking

2019-07-09 Thread Dave Chinner
On Mon, Jul 08, 2019 at 03:31:14PM +0200, Jan Kara wrote: > On Sat 06-07-19 09:31:57, Dave Chinner wrote: > > On Wed, Jul 03, 2019 at 03:04:45AM +0300, Boaz Harrosh wrote: > > > On 20/06/2019 01:37, Dave Chinner wrote: > > > <> > > > > > > > > I'd prefer it doesn't get lifted to the VFS because I'

Re: pagecache locking

2019-07-08 Thread Jan Kara
On Sat 06-07-19 09:31:57, Dave Chinner wrote: > On Wed, Jul 03, 2019 at 03:04:45AM +0300, Boaz Harrosh wrote: > > On 20/06/2019 01:37, Dave Chinner wrote: > > <> > > > > > > I'd prefer it doesn't get lifted to the VFS because I'm planning on > > > getting rid of it in XFS with range locks. i.e. th

Re: pagecache locking

2019-07-07 Thread Dave Chinner
On Sun, Jul 07, 2019 at 06:05:16PM +0300, Boaz Harrosh wrote: > On 06/07/2019 02:31, Dave Chinner wrote: > > > > > As long as the IO ranges to the same file *don't overlap*, it should > > be perfectly safe to take separate range locks (in read or write > > mode) on either side of the mmap_sem as

Re: pagecache locking

2019-07-07 Thread Boaz Harrosh
On 06/07/2019 02:31, Dave Chinner wrote: > > As long as the IO ranges to the same file *don't overlap*, it should > be perfectly safe to take separate range locks (in read or write > mode) on either side of the mmap_sem as non-overlapping range locks > can be nested and will not self-deadlock. >

Re: pagecache locking

2019-07-05 Thread Dave Chinner
On Wed, Jul 03, 2019 at 03:04:45AM +0300, Boaz Harrosh wrote: > On 20/06/2019 01:37, Dave Chinner wrote: > <> > > > > I'd prefer it doesn't get lifted to the VFS because I'm planning on > > getting rid of it in XFS with range locks. i.e. the XFS_MMAPLOCK is > > likely to go away in the near term b

Re: pagecache locking

2019-07-02 Thread Boaz Harrosh
On 03/07/2019 04:07, Patrick Farrell wrote: > Recursively read locking is generally unsafe, that’s why lockdep > complains about it. The common RW lock primitives are queued in > their implementation, meaning this recursive read lock sequence: > P1 - read (gets lock) > P2 - write > P1 - read > >

Re: pagecache locking

2019-07-02 Thread Boaz Harrosh
On 20/06/2019 01:37, Dave Chinner wrote: <> > > I'd prefer it doesn't get lifted to the VFS because I'm planning on > getting rid of it in XFS with range locks. i.e. the XFS_MMAPLOCK is > likely to go away in the near term because a range lock can be > taken on either side of the mmap_sem in the p

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-19 Thread Dave Chinner
On Wed, Jun 19, 2019 at 12:38:38PM +0200, Jan Kara wrote: > On Tue 18-06-19 07:21:56, Amir Goldstein wrote: > > > > Right, but regardless of the spec we have to consider that the > > > > behaviour of XFS comes from it's Irix heritage (actually from EFS, > > > > the predecessor of XFS from the late

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-19 Thread Jan Kara
On Tue 18-06-19 07:21:56, Amir Goldstein wrote: > > > Right, but regardless of the spec we have to consider that the > > > behaviour of XFS comes from it's Irix heritage (actually from EFS, > > > the predecessor of XFS from the late 1980s) > > > > Sure. And as I mentioned, I think it's technically

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-17 Thread Amir Goldstein
> > Right, but regardless of the spec we have to consider that the > > behaviour of XFS comes from it's Irix heritage (actually from EFS, > > the predecessor of XFS from the late 1980s) > > Sure. And as I mentioned, I think it's technically the nicer guarantee. > > That said, it's a pretty *expensi

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-17 Thread Linus Torvalds
On Mon, Jun 17, 2019 at 3:48 PM Dave Chinner wrote: > > The wording of posix changes every time they release a new version > of the standard, and it's _never_ obvious what behaviour the > standard is actually meant to define. They are always written with > sufficient ambiguity and wiggle room that

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-17 Thread Dave Chinner
On Fri, Jun 14, 2019 at 06:01:07PM -1000, Linus Torvalds wrote: > On Thu, Jun 13, 2019 at 5:08 PM Linus Torvalds > wrote: > > > > I do not believe that posix itself actually requires that at all, > > although extended standards may. > > So I tried to see if I could find what this perhaps alludes

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-14 Thread Linus Torvalds
On Thu, Jun 13, 2019 at 5:08 PM Linus Torvalds wrote: > > I do not believe that posix itself actually requires that at all, > although extended standards may. So I tried to see if I could find what this perhaps alludes to. And I suspect it's not in the read/write thing, but the pthreads side tal

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-14 Thread Linus Torvalds
On Thu, Jun 13, 2019 at 9:31 PM Dave Chinner wrote: > > Yes, they do, I see plenty of cases where the page cache works just > fine because it is still faster than most storage. But that's _not > what I said_. I only quoted one small part of your email, because I wanted to point out how you again

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-14 Thread Kent Overstreet
On Fri, Jun 14, 2019 at 09:55:24AM +1000, Dave Chinner wrote: > On Thu, Jun 13, 2019 at 02:36:25PM -0400, Kent Overstreet wrote: > > On Thu, Jun 13, 2019 at 09:02:24AM +1000, Dave Chinner wrote: > > > On Wed, Jun 12, 2019 at 12:21:44PM -0400, Kent Overstreet wrote: > > > > Ok, I'm totally on board

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-14 Thread Dave Chinner
On Thu, Jun 13, 2019 at 04:30:36PM -1000, Linus Torvalds wrote: > On Thu, Jun 13, 2019 at 1:56 PM Dave Chinner wrote: > > > > That said, the page cache is still far, far slower than direct IO, > > Bullshit, Dave. > > You've made that claim before, and it's been complete bullshit before > too, an

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-13 Thread Linus Torvalds
On Thu, Jun 13, 2019 at 1:56 PM Dave Chinner wrote: > > - buffered read and buffered write can run concurrently if they > don't overlap, but right now they are serialised because that's the > only way to provide POSIX atomic write vs read semantics (only XFS > provides userspace with that guarante

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-13 Thread Linus Torvalds
On Thu, Jun 13, 2019 at 1:56 PM Dave Chinner wrote: > > That said, the page cache is still far, far slower than direct IO, Bullshit, Dave. You've made that claim before, and it's been complete bullshit before too, and I've called you out on it then too. Why do you continue to make this obviousl

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-13 Thread Dave Chinner
On Thu, Jun 13, 2019 at 05:21:12PM -0400, Kent Overstreet wrote: > On Thu, Jun 13, 2019 at 03:13:40PM -0600, Andreas Dilger wrote: > > There are definitely workloads that require multiple threads doing > > non-overlapping > > writes to a single file in HPC. This is becoming an increasingly common

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-13 Thread Dave Chinner
On Thu, Jun 13, 2019 at 02:36:25PM -0400, Kent Overstreet wrote: > On Thu, Jun 13, 2019 at 09:02:24AM +1000, Dave Chinner wrote: > > On Wed, Jun 12, 2019 at 12:21:44PM -0400, Kent Overstreet wrote: > > > Ok, I'm totally on board with returning EDEADLOCK. > > > > > > Question: Would we be ok with r

Re: pagecache locking (was: bcachefs status update) merged)

2019-06-13 Thread Kent Overstreet
On Thu, Jun 13, 2019 at 03:13:40PM -0600, Andreas Dilger wrote: > There are definitely workloads that require multiple threads doing > non-overlapping > writes to a single file in HPC. This is becoming an increasingly common > problem > as the number of cores on a single client increase, since t