Re: Suggestion: bmap files and bmaptool

2013-09-17 Thread Artem Bityutskiy
On Sun, 2013-08-18 at 16:43 +0100, Pádraig Brady wrote: > You definitely need the fsync before doing the fiemap. > We saw this on certain file systems including ext4 when adding > fiemap support (efficient reading of holes) to cp. > This is a bug in the fiemap interface IMHO in that it returns > fa

Re: Suggestion: bmap files and bmaptool

2013-09-17 Thread Artem Bityutskiy
On Wed, 2013-08-14 at 12:24 +0200, Björn Persson wrote: > Artem Bityutskiy wrote: > >On Wed, 2013-08-14 at 11:44 +0200, Till Maas wrote: > >> On Wed, Aug 14, 2013 at 12:21:23PM +0300, Artem Bityutskiy wrote: > >> > On Wed, 2013-08-14 at 10:37 +0200, Till Maas wrote: > >> > > On Wed, Aug 14, 2013 at

Re: Suggestion: bmap files and bmaptool

2013-09-17 Thread Artem Bityutskiy
On Wed, 2013-08-14 at 11:44 +0200, Till Maas wrote: > On Wed, Aug 14, 2013 at 12:21:23PM +0300, Artem Bityutskiy wrote: > > On Wed, 2013-08-14 at 10:37 +0200, Till Maas wrote: > > > On Wed, Aug 14, 2013 at 09:31:22AM +0300, Artem Bityutskiy wrote: > > > > > > > Other things like reading from remot

Re: Suggestion: bmap files and bmaptool

2013-08-19 Thread Artem Bityutskiy
On Sun, 2013-08-18 at 16:43 +0100, Pádraig Brady wrote: > You definitely need the fsync before doing the fiemap. > We saw this on certain file systems including ext4 when adding > fiemap support (efficient reading of holes) to cp. > This is a bug in the fiemap interface IMHO in that it returns > fa

Re: Suggestion: bmap files and bmaptool

2013-08-19 Thread Artem Bityutskiy
On Sat, 2013-08-17 at 17:09 +0100, Richard W.M. Jones wrote: > On Thu, Aug 15, 2013 at 12:34:26PM -0500, Eric Sandeen wrote: > > But then there's the issue of transporting these sparse files > > around. We have had the same problem in the past with large e2image > > metadata image files, which may

Re: Suggestion: bmap files and bmaptool

2013-08-19 Thread Artem Bityutskiy
On Fri, 2013-08-16 at 09:51 -0500, Eric Sandeen wrote: > by single file I meant a _single_ file, not the original file and a mapping > file. :) Oh, sorry, OK. > I realize that you have a fully-fledged set of tools, and you're not looking > for new directions, but I was thinking about encoding ma

Re: Suggestion: bmap files and bmaptool

2013-08-18 Thread Pádraig Brady
On 08/16/2013 09:46 AM, Artem Bityutskiy wrote: > Hi Eric, > > thanks for the question. Sorry my answers contain extra information, but > I assume that other people read this, and may benefit from the info. > After all, I am trying to get people interested, this is a good tool > IMO, and I would l

Re: Suggestion: bmap files and bmaptool

2013-08-17 Thread Richard W.M. Jones
On Thu, Aug 15, 2013 at 12:34:26PM -0500, Eric Sandeen wrote: > But then there's the issue of transporting these sparse files > around. We have had the same problem in the past with large e2image > metadata image files, which may be terabytes in length, with only > gigabytes or megabytes of real d

Re: Suggestion: bmap files and bmaptool

2013-08-16 Thread Eric Sandeen
On 8/16/13 3:46 AM, Artem Bityutskiy wrote: >> Another approach which might (?) be more robust, is to somehow encode >> > that sparseness in a single file format that can be >> > transported/compressed/copied w/o losing the sparseness information, >> > and another tool to operate efficiently on tha

Re: Suggestion: bmap files and bmaptool

2013-08-16 Thread Artem Bityutskiy
Hi Eric, thanks for the question. Sorry my answers contain extra information, but I assume that other people read this, and may benefit from the info. After all, I am trying to get people interested, this is a good tool IMO, and I would like to get more users and hopefully contributors. :-) On Th

Re: Suggestion: bmap files and bmaptool

2013-08-15 Thread Eric Sandeen
On 8/13/13 8:58 AM, Artem Bityutskiy wrote: > # Make the image to be sparse > $ cp --sparse=always Fedora-x86_64-19-20130627-sda.raw > Fedora-x86_64-19-20130627-sda.raw.sparse > > # Generate the bmap file > $ bmaptool create Fedora-x86_64-19-20130627-sda.raw.sparse -o > Fedora-x86_64-19-20130627

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Artem Bityutskiy
On Wed, 2013-08-14 at 21:16 -0700, Samuel Sieb wrote: > On 08/14/2013 02:21 AM, Artem Bityutskiy wrote: > > I think I covered this part in the documentation. But here is a short > > description. > > > > 1. The bmap file should be created just after the image is generated. > > 2. The blocks where ze

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Samuel Sieb
On 08/14/2013 02:21 AM, Artem Bityutskiy wrote: I think I covered this part in the documentation. But here is a short description. 1. The bmap file should be created just after the image is generated. 2. The blocks where zeroes were explicitly written will be mapped to real sectors which will co

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Artem Bityutskiy
On Wed, 2013-08-14 at 12:24 +0200, Björn Persson wrote: > Speaking of security, how is the integrity of the bmap file itself > verified? This is not implemented, unfortunately. This is another thing which I probably would need to do, and this is a very good point. I will look at this, after I do

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Björn Persson
Artem Bityutskiy wrote: >On Wed, 2013-08-14 at 11:44 +0200, Till Maas wrote: >> On Wed, Aug 14, 2013 at 12:21:23PM +0300, Artem Bityutskiy wrote: >> > On Wed, 2013-08-14 at 10:37 +0200, Till Maas wrote: >> > > On Wed, Aug 14, 2013 at 09:31:22AM +0300, Artem Bityutskiy wrote: >> > > >> > > > Other

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Artem Bityutskiy
On Wed, 2013-08-14 at 11:44 +0200, Till Maas wrote: > On Wed, Aug 14, 2013 at 12:21:23PM +0300, Artem Bityutskiy wrote: > > On Wed, 2013-08-14 at 10:37 +0200, Till Maas wrote: > > > On Wed, Aug 14, 2013 at 09:31:22AM +0300, Artem Bityutskiy wrote: > > > > > > > Other things like reading from remot

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Till Maas
On Wed, Aug 14, 2013 at 12:21:23PM +0300, Artem Bityutskiy wrote: > On Wed, 2013-08-14 at 10:37 +0200, Till Maas wrote: > > On Wed, Aug 14, 2013 at 09:31:22AM +0300, Artem Bityutskiy wrote: > > > > > Other things like reading from remote sites, progress indicator, > > > protecting your mounted dis

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Artem Bityutskiy
On Wed, 2013-08-14 at 16:35 +0800, Christopher Meng wrote: > 在 2013-8-13 PM9:52,"Artem Bityutskiy" 写道: > > > > Hi Fedora developers, > > > > I would like suggest you to take a look at bmaptool, which you can > use > > for flashing Fedora ISO images to USB sticks (or other block > devices). > > I'v

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Artem Bityutskiy
On Wed, 2013-08-14 at 10:37 +0200, Till Maas wrote: > On Wed, Aug 14, 2013 at 09:31:22AM +0300, Artem Bityutskiy wrote: > > > Other things like reading from remote sites, progress indicator, > > protecting your mounted disks, uncompressing on-the-fly, checking sha1 > > of the data ond of the bmap

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Till Maas
On Wed, Aug 14, 2013 at 09:31:22AM +0300, Artem Bityutskiy wrote: > Other things like reading from remote sites, progress indicator, > protecting your mounted disks, uncompressing on-the-fly, checking sha1 > of the data ond of the bmap file itself - are goodies, although > important ones. Why sha

Re: Suggestion: bmap files and bmaptool

2013-08-14 Thread Christopher Meng
在 2013-8-13 PM9:52,"Artem Bityutskiy" 写道: > > Hi Fedora developers, > > I would like suggest you to take a look at bmaptool, which you can use > for flashing Fedora ISO images to USB sticks (or other block devices). I've read an article about this tool in Chinese months ago. I can help submit a p

Re: Suggestion: bmap files and bmaptool

2013-08-13 Thread Artem Bityutskiy
On Wed, 2013-08-14 at 09:31 +0300, Artem Bityutskiy wrote: > But this all is not the point, these are additional goodies. The main > point of bmaptool is in having the bmap file, and _then_ you get real > speed-up, based on the sparseness of the original image. Just as a rough demonstration, as I

Re: Suggestion: bmap files and bmaptool

2013-08-13 Thread Artem Bityutskiy
On Tue, 2013-08-13 at 17:48 +0200, Jochen Schmitt wrote: > On Tue, Aug 13, 2013 at 04:58:16PM +0300, Artem Bityutskiy wrote: > > Hi Fedora developers, > > > $ dd if=Fedora-x86_64-19-20130627-sda.raw of=/dev/sdd > > 4194304+0 records in > > 4194304+0 records out > > 2147483648 bytes (2.1 GB) copied

Re: Suggestion: bmap files and bmaptool

2013-08-13 Thread Jochen Schmitt
On Tue, Aug 13, 2013 at 04:58:16PM +0300, Artem Bityutskiy wrote: > Hi Fedora developers, > $ dd if=Fedora-x86_64-19-20130627-sda.raw of=/dev/sdd > 4194304+0 records in > 4194304+0 records out > 2147483648 bytes (2.1 GB) copied, 799.487 s, 2.7 MB/s 1.) Seems to be a slow USB flash drive. 2.) Wha

Suggestion: bmap files and bmaptool

2013-08-13 Thread Artem Bityutskiy
Hi Fedora developers, I would like suggest you to take a look at bmaptool, which you can use for flashing Fedora ISO images to USB sticks (or other block devices). Let me set the scene. In order to flash a Fedora ISO image, typically first download the image and then use dd to copy it to the USB