Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC

2018-04-18 Thread Eric Blake
On 04/18/2018 09:25 AM, Vladimir Sementsov-Ogievskiy wrote: >>> Do your code with >>> >>> /* Found an extent, and we're inside it.  */ >>> *next = f.fe.fe_logical + f.fe.fe_length; >>> if (f.fe.fe_flags & FIEMAP_EXTENT_UNWRITTEN) { >>> return BDRV_BLOCK_DATA|BDRV_BLOCK_ZERO;

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2018-04-18 Thread Vladimir Sementsov-Ogievskiy
15.02.2018 19:42, Paolo Bonzini wrote: On 15/02/2018 17:40, Vladimir Sementsov-Ogievskiy wrote: Hi all. Two years later, is there are any news on the topic? I can't understand the following thing:  - FIEMAP without FLAG_SYNC is unsafe  - FIEMAP with FLAG_SYNC is safe but slow  - so, we've

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2018-02-15 Thread Paolo Bonzini
On 15/02/2018 17:40, Vladimir Sementsov-Ogievskiy wrote: > Hi all. > > Two years later, is there are any news on the topic? > > I can't understand the following thing: > >  - FIEMAP without FLAG_SYNC is unsafe >  - FIEMAP with FLAG_SYNC is safe but slow >  - so, we've dropped FIEMAP and use only

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2018-02-15 Thread Vladimir Sementsov-Ogievskiy
Hi all. Two years later, is there are any news on the topic? I can't understand the following thing:  - FIEMAP without FLAG_SYNC is unsafe  - FIEMAP with FLAG_SYNC is safe but slow  - so, we've dropped FIEMAP and use only lseek. So, it means that lseek is safe _and_ fast (at least, faster than

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-22 Thread Paolo Bonzini
> > > i.e. the use of fiemap to duplicate the exact layout of a file > > > from userspace is only posisble if you can /guarantee/ the source > > > file has not changed in any way during the copy operation at the > > > pointin time you finalise the destination data copy. > > > > We don't do exactly

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-22 Thread Dave Chinner
On Thu, Jul 21, 2016 at 10:23:48AM -0400, Paolo Bonzini wrote: > > > 1) avoid copying zero data, to keep the copy process efficient. For this, > > > SEEK_HOLE/SEEK_DATA are enough. > > > > > > 2) copy file contents while preserving the allocation state of the file's > > > extents. > > > > Which

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-21 Thread Paolo Bonzini
> > 1) avoid copying zero data, to keep the copy process efficient. For this, > > SEEK_HOLE/SEEK_DATA are enough. > > > > 2) copy file contents while preserving the allocation state of the file's > > extents. > > Which is /very difficult/ to do safely and reliably. > i.e. the use of fiemap to du

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-21 Thread Dave Chinner
On Thu, Jul 21, 2016 at 01:31:21PM +0100, Pádraig Brady wrote: > On 21/07/16 12:43, Dave Chinner wrote: > > On Wed, Jul 20, 2016 at 03:35:17PM +0200, Niels de Vos wrote: > >> Oh... And I was surprised to learn that "cp" does use FIEMAP and not > >> SEEK_HOLE/SEEK_DATA. You give a strong suggestion

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-21 Thread Pádraig Brady
On 21/07/16 13:41, Dave Chinner wrote: > On Wed, Jul 20, 2016 at 09:40:06AM -0400, Paolo Bonzini wrote: 1) is it expected that SEEK_HOLE skips unwritten extents? >>> >>> There are multiple answers to this, all of which are correct depending >>> on current context and state: >>> >>> 1. No - som

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-21 Thread Dave Chinner
On Wed, Jul 20, 2016 at 09:40:06AM -0400, Paolo Bonzini wrote: > > > 1) is it expected that SEEK_HOLE skips unwritten extents? > > > > There are multiple answers to this, all of which are correct depending > > on current context and state: > > > > 1. No - some filesystems will report clean unwrit

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-21 Thread Pádraig Brady
On 21/07/16 12:43, Dave Chinner wrote: > On Wed, Jul 20, 2016 at 03:35:17PM +0200, Niels de Vos wrote: >> On Wed, Jul 20, 2016 at 10:30:25PM +1000, Dave Chinner wrote: >>> On Wed, Jul 20, 2016 at 05:19:37AM -0400, Paolo Bonzini wrote: Adding ext4 and XFS guys (Lukas and Dave respectively). As

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-21 Thread Dave Chinner
On Wed, Jul 20, 2016 at 03:35:17PM +0200, Niels de Vos wrote: > On Wed, Jul 20, 2016 at 10:30:25PM +1000, Dave Chinner wrote: > > On Wed, Jul 20, 2016 at 05:19:37AM -0400, Paolo Bonzini wrote: > > > Adding ext4 and XFS guys (Lukas and Dave respectively). As a quick > > > recap, the > > > issue he

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-20 Thread Niels de Vos
On Wed, Jul 20, 2016 at 10:30:25PM +1000, Dave Chinner wrote: > On Wed, Jul 20, 2016 at 05:19:37AM -0400, Paolo Bonzini wrote: > > Adding ext4 and XFS guys (Lukas and Dave respectively). As a quick recap, > > the > > issue here is the semantics of FIEMAP and SEEK_HOLE/SEEK_DATA, which we use > >

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-20 Thread Dave Chinner
On Wed, Jul 20, 2016 at 05:19:37AM -0400, Paolo Bonzini wrote: > Adding ext4 and XFS guys (Lukas and Dave respectively). As a quick recap, the > issue here is the semantics of FIEMAP and SEEK_HOLE/SEEK_DATA, which we use in > "qemu-img map". This command prints metadata about a virtual disk > im

Re: [Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-20 Thread Paolo Bonzini
> > 1) is it expected that SEEK_HOLE skips unwritten extents? > > There are multiple answers to this, all of which are correct depending > on current context and state: > > 1. No - some filesystems will report clean unwritten extents as holes. > > 2. Yes - some filesystems will report clean unwr

[Qemu-devel] semantics of FIEMAP without FIEMAP_FLAG_SYNC (was Re: [PATCH v5 13/14] nbd: Implement NBD_CMD_WRITE_ZEROES on server)

2016-07-20 Thread Paolo Bonzini
Adding ext4 and XFS guys (Lukas and Dave respectively). As a quick recap, the issue here is the semantics of FIEMAP and SEEK_HOLE/SEEK_DATA, which we use in "qemu-img map". This command prints metadata about a virtual disk image---which in the case of a raw image amounts to detecting holes and un