Can anyone shed light on the requirements of WAPBL to
access physical sectors on the filesystem device
individually?
The code uses the value 1<
> Please commit.
Sorry but if you think it's necessary for your goal please commit it yourself.
---
Izumi Tsutsui
On Thu, Feb 04, 2010 at 12:15:15AM +0100, Michael van Elst wrote:
> The original adosfs has been used on media with differnet block
> sizes. That doesn't say anything about our implementation, but
> different block sizes have been designed in from the beginning.
I seem to recall having used adosf
On Wed, Feb 03, 2010 at 08:53:14PM +0900, Izumi Tsutsui wrote:
> says:
> >> * A directory consists of some number of blocks of DIRBLKSIZ
> >> * bytes, where DIRBLKSIZ is chosen such that it can be transferred
> >> * to disk in a single atomic operation (e.g. 512 bytes on most machines).
>
> Prob
> > (2) block count stored in dinode
> > (3) DIRBLKSIZ
> >
> > API issue you mentioned is about (1), but what about (2) and (3)
> > that should be defined in file system layer itself?
>
> DIRBLKSIZE it is the chunk size used to access directory 'files'.
> The function ufs_blkatoff() translates th
On Tue, Feb 02, 2010 at 08:19:40AM +, David Holland wrote:
> On Mon, Feb 01, 2010 at 08:58:31PM +, Michael van Elst wrote:
> > The superblock already stores block sizes in terms of byte counts.
> >
> > The problem is that it stores information that shouldn't be there but
> > which depe
On Mon, Feb 01, 2010 at 08:58:31PM +, Michael van Elst wrote:
> The superblock already stores block sizes in terms of byte counts.
>
> The problem is that it stores information that shouldn't be there but
> which depends only on the underlying medium. If you change the medium
> you invali
On Mon, Feb 01, 2010 at 07:48:01PM -0500, Thor Lancelot Simon wrote:
> On Mon, Feb 01, 2010 at 10:59:37PM +, Michael van Elst wrote:
> >
> > It still wouldn't be perfect. Block size is one thing, block alignment
> > the next. A filesystem with 1k blocks (filesystem units == clusters)
> > has p
On Mon, Feb 01, 2010 at 10:59:37PM +, Michael van Elst wrote:
>
> It still wouldn't be perfect. Block size is one thing, block alignment
> the next. A filesystem with 1k blocks (filesystem units == clusters)
> has problems to work on disks with 2k blocks (device units == sectors).
> The only s
mo...@rodents-montreal.org (der Mouse) writes:
>> fs_fsbtodb depends on the block size of the underlying medium.
>Does it? All you need to do is make a mental switch from thinking of
>fs_fsbtodb as "filesystem blocks to underlying device blocks" to
>thinking of it as "filesystem blocks to interch
wo...@planix.ca ("Greg A. Woods") writes:
>Indeed. Or at least using only _one_ logical block size that's
>consistent for the system across all hardware that can be used by the
>system.
Now, that's the easiest, and we already have this (in the kernel).
>However for a portable system I think
>t
> fs_fsbtodb depends on the block size of the underlying medium.
Does it? All you need to do is make a mental switch from thinking of
fs_fsbtodb as "filesystem blocks to underlying device blocks" to
thinking of it as "filesystem blocks to interchange blocks", where an
"interchange block" is defin
On Mon, Feb 01, 2010 at 09:10:05PM +, David Laight wrote:
> On Mon, Feb 01, 2010 at 08:58:31PM +, Michael van Elst wrote:
> >
> > The superblock already stores block sizes in terms of byte counts.
> >
> > The problem is that it stores information that shouldn't be there but
> > which depe
At Mon, 1 Feb 2010 15:34:39 -0500 (EST), der Mouse
wrote:
Subject: Re: blocksizes
>
> > This can easily happen if you copy the image between disks with
> > different block sizes.
>
> Now _there_ is a valid argument for doing everything in terms of bytes
> (as was di
On Mon, Feb 01, 2010 at 08:58:31PM +, Michael van Elst wrote:
>
> The superblock already stores block sizes in terms of byte counts.
>
> The problem is that it stores information that shouldn't be there but
> which depends only on the underlying medium. If you change the medium
> you invalida
mo...@rodents-montreal.org (der Mouse) writes:
>> This can easily happen if you copy the image between disks with
>> different block sizes.
>Now _there_ is a valid argument for doing everything in terms of bytes
>(as was discussed briefly upthread).
The superblock already stores block sizes in t
> This can easily happen if you copy the image between disks with
> different block sizes.
Now _there_ is a valid argument for doing everything in terms of bytes
(as was discussed briefly upthread).
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML
On Mon, Feb 01, 2010 at 10:11:08PM +0900, Izumi Tsutsui wrote:
> Using different translation function might be acceptable, but
> I don't think it's a good idea to ignore fs_fsbtodb value
> stoerd in superblock in kernel ops while userland refers it.
> Can fsck_ffs(8) do proper checks, for example
> > Why do you leave fsbtodb() to use fs_fsbtodb for userland?
> > To have less changes?
>
> Kernel uses DEV_BSIZE devices and needs a proper translations function.
> Userland uses device block sizes and needs a proper translation function.
> Only the translation function (==fsbtodb()) needs to be
On Mon, Feb 01, 2010 at 02:53:20AM +0900, Izumi Tsutsui wrote:
> Why do you leave fsbtodb() to use fs_fsbtodb for userland?
> To have less changes?
Kernel uses DEV_BSIZE devices and needs a proper translations function.
Userland uses device block sizes and needs a proper translation function.
Onl
> >I guess UCB guys put fsbtodb value into superblock to handle
> >variable hardware sector size. But in current implementation
> >some code uses fsbtodb() (ffs_alloccg() etc) and others use
> >btodb() (ffs_alloc() etc) so it's quite inconsistent.
>
> btodb() translates bytes to disk blocks (in DE
tsut...@ceres.dti.ne.jp (Izumi Tsutsui) writes:
>I guess UCB guys put fsbtodb value into superblock to handle
>variable hardware sector size. But in current implementation
>some code uses fsbtodb() (ffs_alloccg() etc) and others use
>btodb() (ffs_alloc() etc) so it's quite inconsistent.
btodb() t
tsut...@ceres.dti.ne.jp (Izumi Tsutsui) writes:
>---
>diff -u src/sys/ufs/ffs/fs.h:1.54 src/sys/ufs/ffs/fs.h:1.55
> :
>@@ -599,8 +599,13 @@
> * Turn file system block numbers into disk block addresses.
> * This maps file system blocks to device size blocks.
> */
>+#if defined (_KERNEL)
>+#defin
> > newfs(8) doesn't work even with -S 2048 option.
> > (probably it tries to write data at offset not sectorsize aligned)
>
> Apparently makefs -S 2048 works, and the resulting image also works only
> when accessed with 2048 byte simulated sector size (fs-utils with ffs
> from rump):
Well, befor
On Sun Jan 31 2010 at 22:21:52 +0900, Izumi Tsutsui wrote:
> > Can you please test with your 2K MO?
>
> It's not easy to test it without working newfs(8) command.
> (if you need hardware I can send the drive and media..)
>
> > N.B. newfs doesn't yet know how to deduce sector sizes, you need
> > t
>
> >To translate file system block size (i.e. unit of fragment size)
> >into raw I/O size, we can ignore fsbtodb.
>
> I have now committed changes to msdofs, ext2fs and ffs to support
> disks with non-512-byte sectors.
>
> Part of that change is a common function to query disk and sector
> size
tsut...@ceres.dti.ne.jp (Izumi Tsutsui) writes:
>To translate file system block size (i.e. unit of fragment size)
>into raw I/O size, we can ignore fsbtodb.
I have now committed changes to msdofs, ext2fs and ffs to support
disks with non-512-byte sectors.
Part of that change is a common function
On Sun, Jan 24, 2010 at 02:37:24PM +0100, Michael van Elst wrote:
> On Sun, Jan 24, 2010 at 11:16:05PM +1030, Brett Lymn wrote:
> > On Fri, Jan 22, 2010 at 01:09:10PM +0100, Michael van Elst wrote:
> > >
> > > Keeping DEV_SIZE at 512 bytes avoids lots of changes.
> > >
> >
> > Won't that mean th
On Thu, Jan 21, 2010 at 10:30:20PM +, Michael van Elst wrote:
> Hello,
>
>
> disk devices are accessed in units of 'blocks', a block can be
> any size, however NetBSD makes assumptions in many places that
> a block is 512 bytes or DEV_BSIZE bytes which makes it impossible
> to use devices wit
On Fri, Jan 22, 2010 at 02:28:07PM +0700, Robert Elz wrote:
> Date:Fri, 22 Jan 2010 08:07:03 +0100
> From:Michael van Elst
> Message-ID: <20100122070702.ga10...@serpens.de>
>
> | except that there a lot of assumptions that
> | physical block size is the same as DE
> > >FYI, Windows XP creates MBR in the first sector even on a removable
> > >2KB/sec MO disk and it seems to use physical block numbers.
> >
> > I haven't found an official spec for MBR on non-512byte blocks, but
> > so far everyone seems to use the first 512 bytes from LBA 0.
>
> That is my sus
> >The design of FFS is independent from design of our buffercache(9)
> >or all disk drivers, so if FFS has its own "disk block size" value
> >in its superblock, we have to check and convert (or reject)
> >the "FFS disk block size value" for our native I/O size
> >whichever we will choose DEV_BSIZE
On Thu, Jan 28, 2010 at 06:54:11PM +, Michael van Elst wrote:
>
> >FYI, Windows XP creates MBR in the first sector even on a removable
> >2KB/sec MO disk and it seems to use physical block numbers.
>
> I haven't found an official spec for MBR on non-512byte blocks, but
> so far everyone seems
tsut...@ceres.dti.ne.jp (Izumi Tsutsui) writes:
>The design of FFS is independent from design of our buffercache(9)
>or all disk drivers, so if FFS has its own "disk block size" value
>in its superblock, we have to check and convert (or reject)
>the "FFS disk block size value" for our native I/O s
> So far I had one remaining question: the fsbtodb shift value.
>
> With a a DEV_BSIZE addressed device you need a shift value
> that is unrelated to the physical sector size and does not
> change when you talk to a device or access an image file.
>
> However, this makes the superblock incompatib
On Thu, Jan 28, 2010 at 04:15:34AM +, David Holland wrote:
> But as you may have noticed, I'm not advocating going back to using
> random mixtures of block sizes.
Well, people are advocating different incompatible ways.
> > It works fine for 16 years now.
>
> ...that is, it works except w
On Mon, Jan 25, 2010 at 08:06:11AM +, Michael van Elst wrote:
> >C hoosing code architectures
>
> 'Redesigning' things to fix bugs seems to be common sense
> nowadays, as if everything existing is always too bad to be
> used.
>
> Of course the same is valid for the redesigned code base
da...@l8s.co.uk (David Laight) writes:
>On Fri, Jan 22, 2010 at 01:09:10PM +0100, Michael van Elst wrote:
>>
>> > - What value should be used for d_secsize in struct disklabel?
>>
>> This should be the physical sector size of the medium (or maybe
>> some multiple so you could have faked but comp
On Fri, Jan 22, 2010 at 01:09:10PM +0100, Michael van Elst wrote:
>
> > - What value should be used for d_secsize in struct disklabel?
>
> This should be the physical sector size of the medium (or maybe
> some multiple so you could have faked but compatible geoemtries).
d_secsize is the units fo
tsut...@ceres.dti.ne.jp (Izumi Tsutsui) writes:
>> cd(4) uses DEV_BSIZE units to address blocks.
>I meant size parameters in struct disklabel, like
>d_secperunit and d_partitions[].p_size etc.
Yes, the disklabel uses physical sector sizes.
>> >Isn't it one example of inconsistent hacks?
>>
>>
> >Disklabel in cd(4) requires hardware block size for partition offsets,
>
> All disklabels use hardware block sizes. cd(4) is no exception.
>
> >but it also uses DEV_BSIZE for sizes.
>
> cd(4) uses DEV_BSIZE units to address blocks.
I meant size parameters in struct disklabel, like
d_secperun
dholland-t...@netbsd.org (David Holland) writes:
>Choosing code architectures
'Redesigning' things to fix bugs seems to be common sense
nowadays, as if everything existing is always too bad to be
used.
Of course the same is valid for the redesigned code base in
the future.
One interesting point
On Sun, Jan 24, 2010 at 11:21:52PM +0100, Michael van Elst wrote:
> > > Not using DEV_BSIZE requires to change how things work now.
> >
> > He is right in the long run, though.
>
> You may think that the way NetBSD works is "a hack" as Izumi Tsutsui
> put it. But the argument that keeping t
tsut...@ceres.dti.ne.jp (Izumi Tsutsui) writes:
>Disklabel in cd(4) requires hardware block size for partition offsets,
All disklabels use hardware block sizes. cd(4) is no exception.
>but it also uses DEV_BSIZE for sizes.
cd(4) uses DEV_BSIZE units to address blocks.
>Isn't it one example of
> This is not true for NetBSD. disks are addressed in units
> of DEV_BSIZE by all filesystems and the sector sizes are
> handled by the disk driver. Even the CD-ROM driver
> is no different.
Disklabel in cd(4) requires hardware block size for partition offsets,
but it also uses DEV_BSIZE for sizes
On Sun, Jan 24, 2010 at 09:54:47PM +, David Holland wrote:
> On Sun, Jan 24, 2010 at 08:48:07PM +0100, Michael van Elst wrote:
> > > A quote, often attributed to Einstein, is, "Everything should be
> > > made as simple as possible, but no simpler." I can't help but feel
> > > that this
On Sun, Jan 24, 2010 at 08:48:32PM +, David Laight wrote:
> > The btodb/dbtob macros will need another argument to indicate
> > where the block size is obtained.
>
> That will just cause massive errors...
> For disks I would go for transfer requests (eg from fs) that are either in
> fixe
On Sun, Jan 24, 2010 at 08:48:07PM +0100, Michael van Elst wrote:
> > A quote, often attributed to Einstein, is, "Everything should be
> > made as simple as possible, but no simpler." I can't help but feel
> > that this is making things simpler then they should be.
>
> John, you misunde
da...@l8s.co.uk (David Laight) writes:
>Forcing the raw disk block size through all the kernel layes will be
>seriously buggy.
To be true, that's how it was 'in the beginning' of Unix and
a rough check shows that FreeBSD still does it that way.
Touching lots of code to go back to this model is
m...@3am-software.com (Matt Thomas) writes:
>> Keeping DEV_SIZE at 512 bytes avoids lots of changes.
>So?
Keeping or changing DEV_BSIZE does nothing for the issue at
hand: support for disks with non-512byte-sectors.
Why would you want to mix a change to DEV_BSIZE for whatever
reason into it?
On Sun, Jan 24, 2010 at 12:21:52PM -0800, Matt Thomas wrote:
>
> My feelings are this: when accessing the device through a bdevsw (d_psize or
> d_strategy), will refer to daddr_t and will be units of the blocksize of the
> device. For block access (aka filesystem) only blocksize that are powers
On Jan 22, 2010, at 4:09 AM, Michael van Elst wrote:
> On Fri, Jan 22, 2010 at 07:36:15PM +0900, Izumi Tsutsui wrote:
>>> disk devices are accessed in units of 'blocks', a block can be
>>> any size, however NetBSD makes assumptions in many places that
>>> a block is 512 bytes or DEV_BSIZE bytes w
On Sun, Jan 24, 2010 at 09:08:10AM -0800, John Nemeth wrote:
> A quote, often attributed to Einstein, is, "Everything should be
> made as simple as possible, but no simpler." I can't help but feel
> that this is making things simpler then they should be.
John, you misunderstand this.
NetBS
On Mon, Jan 25, 2010 at 02:03:48AM +0900, Izumi Tsutsui wrote:
> > N.B. So far I have MSDOSFS and FFS running on a disk with 1K sectors
> > and I learned that the block size translation is already done
> > in our block drivers, so there is no need to funnel I/O through dk.
>
> fdisk(8) and newfs(8
On Jun 16, 9:13am, Michael van Elst wrote:
} On Sun, Jan 24, 2010 at 11:16:05PM +1030, Brett Lymn wrote:
} > On Fri, Jan 22, 2010 at 01:09:10PM +0100, Michael van Elst wrote:
} > >
} > > Keeping DEV_SIZE at 512 bytes avoids lots of changes.
A quote, often attributed to Einstein, is, "Everyt
> N.B. So far I have MSDOSFS and FFS running on a disk with 1K sectors
> and I learned that the block size translation is already done
> in our block drivers, so there is no need to funnel I/O through dk.
fdisk(8) and newfs(8) don't work on 2KB/sector MO drive here.
(though newfs_msdos(8) and moun
On Fri, Jan 22, 2010 at 01:09:10PM +0100, Michael van Elst wrote:
>
> Keeping DEV_SIZE at 512 bytes avoids lots of changes.
>
Won't that mean there is a chance there will be a lot of
read/modify/write going on if the driver is pretending to have 512byte
sectors? Would that not be really really b
On Sun, Jan 24, 2010 at 11:16:05PM +1030, Brett Lymn wrote:
> On Fri, Jan 22, 2010 at 01:09:10PM +0100, Michael van Elst wrote:
> >
> > Keeping DEV_SIZE at 512 bytes avoids lots of changes.
> >
>
> Won't that mean there is a chance there will be a lot of
> read/modify/write going on if the drive
> As an extreme example [on ISO 9660], you could have a file with 3 bytes,
> where every byte is in a separate block.
Raising the question of which kind of resource limitation exactly you want to
impose on the user. Wouldn't it make sense to count that kind of usage as three
blocks? After all, it
On Fri, Jan 22, 2010 at 07:36:15PM +0900, Izumi Tsutsui wrote:
> > disk devices are accessed in units of 'blocks', a block can be
> > any size, however NetBSD makes assumptions in many places that
> > a block is 512 bytes or DEV_BSIZE bytes which makes it impossible
> > to use devices with differen
> disk devices are accessed in units of 'blocks', a block can be
> any size, however NetBSD makes assumptions in many places that
> a block is 512 bytes or DEV_BSIZE bytes which makes it impossible
> to use devices with different block sizes.
>
> IMHO there need to be three different ways to speci
Hi,
One place where that's currently not true is in the quote files,
there block counts are in blocks, and are 32 bits - making the counts
64 bits would double the size of the quota files (the entries really
want to remain a power of two bytes big), and halve cache efficiency
when reading them.
On Fri, Jan 22, 2010 at 08:07:03AM +0100, Michael van Elst wrote:
> On Fri, Jan 22, 2010 at 05:46:31AM +, David Holland wrote:
> > On Thu, Jan 21, 2010 at 10:30:20PM +, Michael van Elst wrote:
> > > IMHO there need to be three different ways to specify block
> > > offsets and block coun
On Fri, Jan 22, 2010 at 07:38:14AM +, Michael van Elst wrote:
> >Like most things, there is no universal correct answer here, simply
> >deciding "always use bytes" because it seems simpler is unlikely to be
> >the overall best answer.
>
> I think the suggestion is to use block numbers (or
jo...@britannica.bec.de (Joerg Sonnenberger) writes:
>For FFS2 it doesn't matter ATM, because
>it is broken in that regard anyway, I think.
Fix it...
--
--
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk
On Fri, Jan 22, 2010 at 02:28:07PM +0700, Robert Elz wrote:
> Date:Fri, 22 Jan 2010 08:07:03 +0100
> From:Michael van Elst
> Message-ID: <20100122070702.ga10...@serpens.de>
>
> | except that there a lot of assumptions that
> | physical block size is the same as DE
k...@munnari.oz.au (Robert Elz) writes:
>Like most things, there is no universal correct answer here, simply
>deciding "always use bytes" because it seems simpler is unlikely to be
>the overall best answer.
I think the suggestion is to use block numbers (or some other form
of addressing larger un
On Fri, Jan 22, 2010 at 02:06:27PM +0700, Robert Elz wrote:
> One place where that's currently not true is in the quote files,
> there block counts are in blocks, and are 32 bits
For FFS/UFS v1 it doesn't matter as filesystem blocks are 32bit and
relative to the fragment size. For FFS2 it doesn't
Date:Fri, 22 Jan 2010 08:07:03 +0100
From:Michael van Elst
Message-ID: <20100122070702.ga10...@serpens.de>
| except that there a lot of assumptions that
| physical block size is the same as DEV_BSIZE.
Is that really true these days? I believe there are assumpti
On Fri, Jan 22, 2010 at 05:46:31AM +, David Holland wrote:
> On Thu, Jan 21, 2010 at 10:30:20PM +, Michael van Elst wrote:
> > IMHO there need to be three different ways to specify block
> > offsets and block counts:
> >
> > 1. in units of blocks of the physical device
> > 2. in units
Date:Fri, 22 Jan 2010 05:46:31 +
From:David Holland
Message-ID: <20100122054631.gc23...@netbsd.org>
| Since in practice nothing can be larger than the maximum value of
| off_t anyway, and all counts should be getting carried around as
| 64-bit values, using
On Thu, Jan 21, 2010 at 10:30:20PM +, Michael van Elst wrote:
> IMHO there need to be three different ways to specify block
> offsets and block counts:
>
> 1. in units of blocks of the physical device
> 2. in units of blocks of DEV_BSIZE bytes
> 3. in bytes
Don't forget: 4. in units of
Hello,
disk devices are accessed in units of 'blocks', a block can be
any size, however NetBSD makes assumptions in many places that
a block is 512 bytes or DEV_BSIZE bytes which makes it impossible
to use devices with different block sizes.
IMHO there need to be three different ways to specify
73 matches
Mail list logo
