On Mon, 26 Feb 2007 06:49:05 -0500, Yakov Lerner <[EMAIL PROTECTED]> wrote:
On 2/14/07, sfaibish <[EMAIL PROTECTED]> wrote:
On Sat, 10 Feb 2007 22:06:37 -0500, Sorin Faibish <[EMAIL PROTECTED]>
wrote:
> Introducing DualFS
>
> File System developers played with the idea of separation of
> met
Yakov Lerner wrote:
> On 2/14/07, sfaibish <[EMAIL PROTECTED]> wrote:
>> On Sat, 10 Feb 2007 22:06:37 -0500, Sorin Faibish <[EMAIL PROTECTED]>
>> wrote:
>>
>> > Introducing DualFS
>> >
>> > File System developers played with the idea of separation of
>> > meta-data from data in file systems for a w
On 2/14/07, sfaibish <[EMAIL PROTECTED]> wrote:
On Sat, 10 Feb 2007 22:06:37 -0500, Sorin Faibish <[EMAIL PROTECTED]> wrote:
> Introducing DualFS
>
> File System developers played with the idea of separation of
> meta-data from data in file systems for a while. The idea was
> lately revived by a
On Sun, 25 Feb 2007, [utf-8] Jörn Engel wrote:
On Sun, 25 February 2007 03:41:40 +0100, Juan Piernas Canovas wrote:
Well, our experimental results say another thing. As I have said, the
greatest part of the files are written at once, so their meta-data blocks
are together on disk. This allows
On Sun, 25 February 2007 03:41:40 +0100, Juan Piernas Canovas wrote:
>
> Well, our experimental results say another thing. As I have said, the
> greatest part of the files are written at once, so their meta-data blocks
> are together on disk. This allows DualFS to implement an explicit
> prefet
Hi Jörn,
On Fri, 23 Feb 2007, [utf-8] Jörn Engel wrote:
On Thu, 22 February 2007 20:57:12 +0100, Juan Piernas Canovas wrote:
I do not agree with this picture, because it does not show that all the
indirect blocks which point to a direct block are along with it in the
same segment. That figure
Jorg,
I am very found of all your comments and your positive attitude
on DualFS. I also understand that you have much more experience
than us in regard to GC and "cleaners". DualFS implementation is
using maybe old technology that can be definetly improved. Although
we understand the value of Dua
On Thu, 22 February 2007 20:57:12 +0100, Juan Piernas Canovas wrote:
>
> I do not agree with this picture, because it does not show that all the
> indirect blocks which point to a direct block are along with it in the
> same segment. That figure should look like:
>
> Segment 1: [some data] [ DA
Hi Jörn,
On Thu, 22 Feb 2007, [utf-8] Jörn Engel wrote:
A partial segment is a transaction unit, and contains "all" the blocks
modified by a file system operation, including indirect blocks and i-nodes
(actually, it contains the blocks modified by several file system
operations, but let us assu
On Thu, 22 February 2007 05:30:03 +0100, Juan Piernas Canovas wrote:
>
> DualFS writes meta-blocks in variable-sized chunks that we call partial
> segments. The meta-data device, however, is divided into segments, which
> have the same size. A partial segment can be as large a a segment, but a
Hi Jörn,
I have been thinking about the problem that you describe, and,
definitively, DualFS does not have that problem. I could be wrong, but,
I actually believe that the GC implemented by DualFS is deadlock-free.
The key is the design of the log-structured file system used by DualFS for
the
On Wed, 21 February 2007 19:31:40 +0100, Juan Piernas Canovas wrote:
>
> I do not understand. Do you mean that if I have 10 segments, 5 busy and 5
> free, after cleaning I could need 6 segments? How? Where the extra blocks
> come from?
This is a fairly complicated subject and I have trouble exp
Hi Jörn,
On Wed, 21 Feb 2007, [utf-8] Jörn Engel wrote:
On Wed, 21 February 2007 05:36:22 +0100, Juan Piernas Canovas wrote:
I don't see how you can guarantee 50% free segments. Can you explain
that bit?
It is quite simple. If 50% of your segments are busy, and the other 50%
are free, and t
On Wed, 21 February 2007 05:36:22 +0100, Juan Piernas Canovas wrote:
> >
> >I don't see how you can guarantee 50% free segments. Can you explain
> >that bit?
> It is quite simple. If 50% of your segments are busy, and the other 50%
> are free, and the file system needs a new segment, the cleaner
Hi Jörn,
On Tue, 20 Feb 2007, [utf-8] Jörn Engel wrote:
On Tue, 20 February 2007 00:57:50 +0100, Juan Piernas Canovas wrote:
Actually, the GC may become a problem when the number of free segments is
50% or less. If your LFS always guarantees, at least, 50% of free
"segments" (note that I am t
Juan Piernas Canovas wrote:
The point of all the above is that you must improve the common case,
and manage the worst case correctly.
That statement made it to my quote file. Of course "correctly" hopefully
means getting to the desired behavior without a performance hit so bad
it becomes a "
On Tue, 20 February 2007 00:57:50 +0100, Juan Piernas Canovas wrote:
>
> I understand the problem that you describe with respect to the GC, but
> let me explain why I think that it has a small impact on DualFS.
>
> Actually, the GC may become a problem when the number of free segments is
> 50%
On Tue, Feb 20, 2007 at 12:57:50AM +0100, Juan Piernas Canovas wrote:
> Now, let us assume that the data device takes 90% of the disk space, and
> the meta-data device the other 10%. When the data device gets full, the
> meta-data blocks will be using the half of the meta-data device, and the
>
Hi Jörn,
I understand the problem that you describe with respect to the GC, but
let me explain why I think that it has a small impact on DualFS.
Actually, the GC may become a problem when the number of free segments is
50% or less. If your LFS always guarantees, at least, 50% of free
"segmen
Maybe this is a decent approach to deal with the problem. First some
definitions. T is the target segment to be cleaned, S is the spare
segment that valid data is written to, O are other segments that contain
indirect blocks I for valid data D in T.
Have two different GC mechanisms to choose bet
On Sat, 17 February 2007 15:47:01 -0500, Sorin Faibish wrote:
>
> DualFS can probably get around this corner case as it is up to the user
> to select the size of the MD device size. If you want to prevent this
> corner case you can always use a device bigger than 10% of the data device
> which is e
On Sat, 17 Feb 2007 13:36:46 -0500, Jörn Engel <[EMAIL PROTECTED]>
wrote:
On Sat, 17 February 2007 13:10:23 -0500, Bill Davidsen wrote:
>
I missed that. Which corner case did you find triggers this in DualFS?
This is not specific to DualFS, it applies to any log-structured
filesystem.
Garb
On Sat, 17 February 2007 13:10:23 -0500, Bill Davidsen wrote:
> >
> I missed that. Which corner case did you find triggers this in DualFS?
This is not specific to DualFS, it applies to any log-structured
filesystem.
Garbage collection always needs at least one spare segment to collect
valid dat
Jörn Engel wrote:
On Fri, 16 February 2007 18:47:48 -0500, Bill Davidsen wrote:
Actually I am interested in the common case, where the machine is not
out of space, or memory, or CPU, but when it is appropriately sized to
the workload. Not that I lack interest in corner cases, but the "runnin
On Fri, 16 February 2007 18:47:48 -0500, Bill Davidsen wrote:
> >
> Actually I am interested in the common case, where the machine is not
> out of space, or memory, or CPU, but when it is appropriately sized to
> the workload. Not that I lack interest in corner cases, but the "running
> flat out
Jörn Engel wrote:
On Thu, 15 February 2007 23:59:14 +0100, Juan Piernas Canovas wrote:
Actually, the version of DualFS for Linux 2.4.19 implements a cleaner. In
our case, the cleaner is not really a problem because there is not too
much to clean (the meta-data device only contains meta-data blo
> "JE" == Jörn Engel <[EMAIL PROTECTED]> writes:
JE> Being good where log-structured filesystems usually are horrible
JE> is a challenge. And I'm sure many people are more interested in
JE> those performance number than in the ones you shine at. :)
Anything that helps performance when untarri
On Thu, 15 February 2007 23:59:14 +0100, Juan Piernas Canovas wrote:
> >
> Actually, the version of DualFS for Linux 2.4.19 implements a cleaner. In
> our case, the cleaner is not really a problem because there is not too
> much to clean (the meta-data device only contains meta-data blocks which
Hi,
On Fri, 16 Feb 2007, Andi Kleen wrote:
If you stripe two disks with a standard fs versus use one of them
as metadata volume and the other as data volume with dualfs i would
expect the striped variant usually be faster because it will give
parallelism not only to data versus metadata, but al
On Thu, 15 Feb 2007 14:46:34 -0500, Jan Engelhardt
<[EMAIL PROTECTED]> wrote:
On Feb 15 2007 21:38, Andi Kleen wrote:
Also I would expect your design to be slow for metadata read intensive
workloads. E.g. have you tried to boot a root partition with dual fs?
That's a very important IO bench
> >Also many storage subsystems have some internal parallelism
> >in writing (e.g. a RAID can write on different disks in parallel for
> >a single partition) so i'm not sure your distinction is that useful.
> >
> But we are talking about a different case. What I have said is that if you
> use two
Hi Jörn,
On Thu, 15 Feb 2007, [utf-8] Jörn Engel wrote:
On Thu, 15 February 2007 19:38:14 +0100, Juan Piernas Canovas wrote:
The patch for 2.6.11 is not still stable enough to be released. Be patient
;-)
While I don't want to discourage you, this is about the point in
development where most
Hi Andi,
On Thu, 15 Feb 2007, Andi Kleen wrote:
Juan Piernas Canovas <[EMAIL PROTECTED]> writes:
[playing devil's advocate here]
If the data and meta-data devices of DualFS can be on different disks,
DualFS is able to READ and WRITE data and meta-data blocks in
PARALLEL.
XFS can do this to
On Thu, 15 February 2007 19:38:14 +0100, Juan Piernas Canovas wrote:
>
> The patch for 2.6.11 is not still stable enough to be released. Be patient
> ;-)
While I don't want to discourage you, this is about the point in
development where most log structured filesystems stopped. Doing a
little we
On Feb 15 2007 21:38, Andi Kleen wrote:
>
>Also I would expect your design to be slow for metadata read intensive
>workloads. E.g. have you tried to boot a root partition with dual fs?
>That's a very important IO benchmark for desktop Linux systems.
Did someone say metadata intensive? Try kernel
Juan Piernas Canovas <[EMAIL PROTECTED]> writes:
[playing devil's advocate here]
> If the data and meta-data devices of DualFS can be on different disks,
> DualFS is able to READ and WRITE data and meta-data blocks in
> PARALLEL.
XFS can do this too using its real time volumes (which don't cont
Hi all,
On Wed, 14 Feb 2007, Jan Engelhardt wrote:
On Feb 14 2007 16:10, sfaibish wrote:
1. DualFS has only one copy of every meta-data block. This copy is
in the meta-data device,
Where does this differ from typical filesystems like xfs?
At least ext3 and xfs have an option to store the l
On Feb 14 2007 16:10, sfaibish wrote:
>>
>> 1. DualFS has only one copy of every meta-data block. This copy is
>> in the meta-data device,
Where does this differ from typical filesystems like xfs?
At least ext3 and xfs have an option to store the log/journal
on another device too.
>> The DualFS
On Sat, 10 Feb 2007 22:06:37 -0500, Sorin Faibish <[EMAIL PROTECTED]> wrote:
Introducing DualFS
File System developers played with the idea of separation of
meta-data from data in file systems for a while. The idea was
lately revived by a small group of file system enthusiasts
from Spain (from
39 matches
Mail list logo
