[2/4] DST: Core distributed storage files.

2008-01-22 Thread Evgeniy Polyakov
Core distributed storage files. Include userspace interfaces, initialization, block layer bindings and other core functionality. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index b4c8319..ca6592d 100644 --- a/drivers

[4/4] DST: Algorithms used in distributed storage.

2008-01-22 Thread Evgeniy Polyakov
Algorithms used in distributed storage. Mirror and linear mapping code. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/dst/alg_linear.c b/drivers/block/dst/alg_linear.c new file mode 100644 index 000..2f9ed65 --- /dev/null +++ b/drivers/blo

[0/4] DST: Distributed storage: Succumbed to live ant.

2008-01-22 Thread Evgeniy Polyakov
Distributed storage: Succumbed to live ant. I'm pleased to announce the 14'th release of the distributed storage subsystem (DST). DST allows to form a storage on top of local and remote nodes and combine them into linear or mirroring setup, which in turn can be exported to remote no

[1/4] DST: Distributed storage documentation.

2008-01-22 Thread Evgeniy Polyakov
Distributed storage documentation. Algorithms used in the system, userspace interfaces (sysfs dirs and files), design and implementation details are described here. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/Documentation/dst/algorithms.txt b/Documentati

[4/4] DST: Algorithms used in distributed storage.

2007-12-26 Thread Evgeniy Polyakov
Algorithms used in distributed storage. Mirror and linear mapping code. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/dst/alg_linear.c b/drivers/block/dst/alg_linear.c new file mode 100644 index 000..2f9ed65 --- /dev/null +++ b/drivers/blo

[1/4] DST: Distributed storage documentation.

2007-12-26 Thread Evgeniy Polyakov
Distributed storage documentation. Algorithms used in the system, userspace interfaces (sysfs dirs and files), design and implementation details are described here. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/Documentation/dst/algorithms.txt b/Documentati

[0/4] DST: Distributed storage: Groundhogs strike back: no New Year for humans

2007-12-26 Thread Evgeniy Polyakov
Distributed storage: Groundhogs strike back: no New Year for humans I'm pleased to announce the 13'th release of the distributed storage subsystem (DST). DST allows to form a storage on top of local and remote nodes and combine them into linear or mirroring setup, which in turn can b

[2/4] DST: Core distributed storage files.

2007-12-26 Thread Evgeniy Polyakov
Core distributed storage files. Include userspace interfaces, initialization, block layer bindings and other core functionality. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index b4c8319..ca6592d 100644 --- a/drivers

Re: [0/4] DST: Distributed storage.

2007-12-18 Thread Evgeniy Polyakov
Hi David. On Tue, Dec 18, 2007 at 12:00:04PM +1100, David Chinner ([EMAIL PROTECTED]) wrote: > On Mon, Dec 17, 2007 at 06:03:38PM +0300, Evgeniy Polyakov wrote: > > DST passed all FS tests in LTP with XFS (modulo MAX_LOCK_DEPTH too low bug: > > [ 8398.605691] BUG: MAX_LOCK_DEPTH too low! > > [ 83

Re: [0/4] DST: Distributed storage.

2007-12-17 Thread David Chinner
On Mon, Dec 17, 2007 at 06:03:38PM +0300, Evgeniy Polyakov wrote: > DST passed all FS tests in LTP with XFS (modulo MAX_LOCK_DEPTH too low bug: > [ 8398.605691] BUG: MAX_LOCK_DEPTH too low! > [ 8398.609641] turning off the locking correctness validator. Evgeniy, can you please start reporting thes

Re: [1/4] DST: Distributed storage documentation.

2007-12-17 Thread Kay Sievers
On Dec 17, 2007 4:03 PM, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > +++ b/Documentation/dst/sysfs.txt > @@ -0,0 +1,33 @@ > +This file describes sysfs files created for each storage. > + > +1. Per-storage files. > +Each storage has its own dir /sysfs/devices/$storage_name, > +2. Per-node files.

[1/4] DST: Distributed storage documentation.

2007-12-17 Thread Evgeniy Polyakov
Distributed storage documentation. Algorithms used in the system, userspace interfaces (sysfs dirs and files), design and implementation details are described here. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/Documentation/dst/algorithms.txt b/Documentati

[4/4] DST: Algorithms used in distributed storage.

2007-12-17 Thread Evgeniy Polyakov
Algorithms used in distributed storage. Mirror and linear mapping code. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/dst/alg_linear.c b/drivers/block/dst/alg_linear.c new file mode 100644 index 000..836764d --- /dev/null +++ b/drivers/blo

[2/4] DST: Core distributed storage files.

2007-12-17 Thread Evgeniy Polyakov
Core distributed storage files. Include userspace interfaces, initialization, block layer bindings and other core functionality. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index b4c8319..ca6592d 100644 --- a/drivers

[0/4] DST: Distributed storage.

2007-12-17 Thread Evgeniy Polyakov
Distributed storage. I'm pleased to announce the 12'th release of the distributed storage subsystem (DST). DST allows to form a storage on top of local and remote nodes and combine them into linear or mirroring setup, which in turn can be exported to remote nodes. Short changel

Re: [4/4] DST: Algorithms used in distributed storage.

2007-12-12 Thread Evgeniy Polyakov
On Wed, Dec 12, 2007 at 12:12:47PM +0300, Dmitry Monakhov ([EMAIL PROTECTED]) wrote: > On 14:47 Mon 10 Dec , Evgeniy Polyakov wrote: > > > > Algorithms used in distributed storage. > > Mirror and linear mapping code. > Hi, i've finally take a look on your DST s

Re: [4/4] DST: Algorithms used in distributed storage.

2007-12-12 Thread Dmitry Monakhov
On 14:47 Mon 10 Dec , Evgeniy Polyakov wrote: > > Algorithms used in distributed storage. > Mirror and linear mapping code. Hi, i've finally take a look on your DST solution. It seems what your current implementation will not work on nonstandard devices for example softwar

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Evgeniy Polyakov
hat? At least > > > > > > not > > > > > > in 2.6.22 what I'm using for now. > > > > > > > > > > Yeah, a loop would look pretty ugly. How about just adding kref_set(), > > > > > if you need it. > > > &g

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Kay Sievers
gt; Yeah, a loop would look pretty ugly. How about just adding kref_set(), > > > > if you need it. > > > > > > Well, then it distributed storage will not be able to build as > > > standalone module, and kref_set() itself will not be accepted as a single &

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Evgeniy Polyakov
ry ugly step. Is there kref_set() or somethinglike that? At least not > > > > in 2.6.22 what I'm using for now. > > > > > > Yeah, a loop would look pretty ugly. How about just adding kref_set(), > > > if you need it. > > > > Well, then it distributed s

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Kay Sievers
ref_get() in a loop, which is a > > > very ugly step. Is there kref_set() or somethinglike that? At least not > > > in 2.6.22 what I'm using for now. > > > > Yeah, a loop would look pretty ugly. How about just adding kref_set(), > > if you need it. > >

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Evgeniy Polyakov
> > Yeah, a loop would look pretty ugly. How about just adding kref_set(), > if you need it. Well, then it distributed storage will not be able to build as standalone module, and kref_set() itself will not be accepted as a single patch, since there are no in-kernel users :) It is

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Kay Sievers
On Mon, 2007-12-10 at 17:50 +0300, Evgeniy Polyakov wrote: > On Mon, Dec 10, 2007 at 03:31:48PM +0100, Kay Sievers ([EMAIL PROTECTED]) > wrote: > > > I meant that for each new device, it will be placed into > > > /sys/devices/its_name, but it can also be accessed via > > > /sys/bus/dst/devices/ >

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Evgeniy Polyakov
On Mon, Dec 10, 2007 at 05:50:55PM +0300, Evgeniy Polyakov ([EMAIL PROTECTED]) wrote: > > Further questions: > > Why do you do your own refcounting instead of using kref? > > That's because I always used atomic operations as a reference counters > and did not tried krefs :) > They are the same ac

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Evgeniy Polyakov
On Mon, Dec 10, 2007 at 03:31:48PM +0100, Kay Sievers ([EMAIL PROTECTED]) wrote: > > I meant that for each new device, it will be placed into > > /sys/devices/its_name, but it can also be accessed via > > /sys/bus/dst/devices/ > > Still, it looks like a path. :) > > Please don't reference any dev

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Kay Sievers
On Mon, 2007-12-10 at 15:58 +0300, Evgeniy Polyakov wrote: > On Mon, Dec 10, 2007 at 01:51:43PM +0100, Kay Sievers ([EMAIL PROTECTED]) > wrote: > > On Dec 10, 2007 12:47 PM, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > diff --git a/Documentation/dst/sysfs.txt b/Documentation/dst/sysfs.txt > >

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Evgeniy Polyakov
On Mon, Dec 10, 2007 at 01:51:43PM +0100, Kay Sievers ([EMAIL PROTECTED]) wrote: > On Dec 10, 2007 12:47 PM, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > diff --git a/Documentation/dst/sysfs.txt b/Documentation/dst/sysfs.txt > > new file mode 100644 > > index 000..79d79dc > > --- /dev/null >

Re: [1/4] DST: Distributed storage documentation.

2007-12-10 Thread Kay Sievers
On Dec 10, 2007 12:47 PM, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > diff --git a/Documentation/dst/sysfs.txt b/Documentation/dst/sysfs.txt > new file mode 100644 > index 000..79d79dc > --- /dev/null > +++ b/Documentation/dst/sysfs.txt > @@ -0,0 +1,30 @@ > +This file describes sysfs files cr

[4/4] DST: Algorithms used in distributed storage.

2007-12-10 Thread Evgeniy Polyakov
Algorithms used in distributed storage. Mirror and linear mapping code. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/dst/alg_linear.c b/drivers/block/dst/alg_linear.c new file mode 100644 index 000..9dc0976 --- /dev/null +++ b/drivers/blo

[2/4] DST: Core distributed storage files.

2007-12-10 Thread Evgeniy Polyakov
Core distributed storage files. Include userspace interfaces, initialization, block layer bindings and other core functionality. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index b4c8319..ca6592d 100644 --- a/drivers

[1/4] DST: Distributed storage documentation.

2007-12-10 Thread Evgeniy Polyakov
Distributed storage documentation. Algorithms used in the system, userspace interfaces (sysfs dirs and files), design and implementation details are described here. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/Documentation/dst/algorithms.txt b/Documentati

[0/4] DST: Distributed storage.

2007-12-10 Thread Evgeniy Polyakov
Distributed storage. I'm pleased to announce the 11'th release of the distributed storage subsystem (DST). This is a maintenance release and includes bug fixes and simple feature extensions only. DST allows to form a storage on top of local and remote nodes and combine them into

Re: [0/4] DST: Distributed storage.

2007-12-04 Thread Evgeniy Polyakov
On Tue, Dec 04, 2007 at 04:56:26PM +, Christoph Hellwig ([EMAIL PROTECTED]) wrote: > > > * fixed bug with XFS metadata update (it can provide slab pages to the > > > DST, so it is not allowed to transfer them using ->sendpage()) > > xfs hasn't been doing that anymore for quite a whil

Re: [0/4] DST: Distributed storage.

2007-12-04 Thread Christoph Hellwig
> > * fixed bug with XFS metadata update (it can provide slab pages to the > > DST, so it is not allowed to transfer them using ->sendpage()) xfs hasn't been doing that anymore for quite a while. Block drivers don't need hacks for it anymore, epsecially as it's not reliably detectable.

Re: [0/4] DST: Distributed storage.

2007-12-04 Thread Evgeniy Polyakov
Hi Mike. On Tue, Dec 04, 2007 at 10:25:29AM -0500, Mike Snitzer ([EMAIL PROTECTED]) wrote: > Thanks for your continued work on DST. I'd like to know if you've > thought further about how synchronous mirroring would be best > implemented with DST. > > You shared you views some time ago via comme

Re: [0/4] DST: Distributed storage.

2007-12-04 Thread Mike Snitzer
On Dec 4, 2007 9:37 AM, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > Distributed storage. > > I'm pleased to announce the 10'th release of the distributed > storage subsystem (DST). This is a maintenance release and includes > bug fixes and simple feature exten

[2/4] DST: Core distributed storage files.

2007-12-04 Thread Evgeniy Polyakov
Core distributed storage files. Include userspace interfaces, initialization, block layer bindings and other core functionality. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index b4c8319..ca6592d 100644 --- a/drivers

[1/4] DST: Distributed storage documentation.

2007-12-04 Thread Evgeniy Polyakov
Distributed storage documentation. Algorithms used in the system, userspace interfaces (sysfs dirs and files), design and implementation details are described here. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/Documentation/dst/algorithms.txt b/Documentati

[0/4] DST: Distributed storage.

2007-12-04 Thread Evgeniy Polyakov
Distributed storage. I'm pleased to announce the 10'th release of the distributed storage subsystem (DST). This is a maintenance release and includes bug fixes and simple feature extensions only. DST allows to form a storage on top of local and remote nodes and combine them into

[4/4] DST: Algorithms used in distributed storage.

2007-12-04 Thread Evgeniy Polyakov
Algorithms used in distributed storage. Mirror and linear mapping code. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/dst/alg_linear.c b/drivers/block/dst/alg_linear.c new file mode 100644 index 000..cb77b57 --- /dev/null +++ b/drivers/blo

Re: [1/4] dst: Distributed storage documentation.

2007-12-03 Thread Evgeniy Polyakov
Hi Matt. On Sun, Dec 02, 2007 at 10:50:59PM -0600, Matt Mackall ([EMAIL PROTECTED]) wrote: > > Distributed storage documentation. > > > > Algorithms used in the system, userspace interfaces > > (sysfs dirs and files), design and implementation details > > are des

Re: [1/4] dst: Distributed storage documentation.

2007-12-02 Thread Matt Mackall
On Thu, Nov 29, 2007 at 03:53:23PM +0300, Evgeniy Polyakov wrote: > > Distributed storage documentation. > > Algorithms used in the system, userspace interfaces > (sysfs dirs and files), design and implementation details > are described here. Can you give us a summary of how

[2/4] dst: Core distributed storage files.

2007-11-29 Thread Evgeniy Polyakov
Core distributed storage files. Include userspace interfaces, initialization, block layer bindings and other core functionality. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index b4c8319..ca6592d 100644 --- a/drivers

[1/4] dst: Distributed storage documentation.

2007-11-29 Thread Evgeniy Polyakov
Distributed storage documentation. Algorithms used in the system, userspace interfaces (sysfs dirs and files), design and implementation details are described here. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/Documentation/dst/algorithms.txt b/Documentati

[4/4] dst: Algorithms used in distributed storage.

2007-11-29 Thread Evgeniy Polyakov
Algorithms used in distributed storage. Mirror and linear mapping code. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/dst/alg_linear.c b/drivers/block/dst/alg_linear.c new file mode 100644 index 000..cb77b57 --- /dev/null +++ b/drivers/blo

[0/4] dst: Distributed storage.

2007-11-29 Thread Evgeniy Polyakov
Distributed storage. I'm pleased to announce the 9'th release of the distributed storage subsystem (DST). This is maintenance release and include bug fixing only. DST allows to form a storage on top of local and remote nodes and combine them into linear or mirroring setup, which in t

[take8 4/4] dst: Algorithms used in distributed storage.

2007-11-20 Thread Evgeniy Polyakov
Algorithms used in distributed storage. Mirror and linear mapping code. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/dst/alg_linear.c b/drivers/block/dst/alg_linear.c new file mode 100644 index 000..cb77b57 --- /dev/null +++ b/drivers/blo

[take8 0/4] dst: Distributed storage.

2007-11-20 Thread Evgeniy Polyakov
Distributed storage. I'm pleased to announce the 8'th release of the distributed storage subsystem (DST). This is a maintenance release and includes bug fixes only. DST allows to form a storage on top of local and remote nodes and combine them into linear or mirroring setup, which i

[take8 2/4] dst: Core distributed storage files.

2007-11-20 Thread Evgeniy Polyakov
Core distributed storage files. Include userspace interfaces, initialization, block layer bindings and other core functionality. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig index b4c8319..ca6592d 100644 --- a/drivers

[take8 1/4] dst: Distributed storage documentation.

2007-11-20 Thread Evgeniy Polyakov
Distributed storage documentation. Algorithms used in the system, userspace interfaces (sysfs dirs and files), design and implementation details are described here. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/Documentation/dst/algorithms.txt b/Documentati

[3/4] Distributed storage. Algorithms.

2007-11-05 Thread Evgeniy Polyakov
Mirror and linear data stripping algorithms for DST. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/dst/alg_linear.c b/drivers/block/dst/alg_linear.c new file mode 100644 index 000..cb77b57 --- /dev/null +++ b/drivers/block/dst/alg_linear.c @@ -0,0 +1,104 @@ +/

[4/4] Distributed storage. Core interfaces.

2007-11-05 Thread Evgeniy Polyakov
This one contains core interfaces of the distributed storage, storage and node initialization and cleanup code, block layer callbacks and the like. It also contains Kconfig and Makefile changes. Signed-off-by: Evgeniy Polyakov <[EMAIL PROTECTED]> diff --git a/drivers/block/Kconfig b/d

[2/4] Distributed storage. Network processing.

2007-11-05 Thread Evgeniy Polyakov
%s: Client: " + "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x:%d", + __func__, + NIP6(sin->sin6_addr), ntohs(sin->sin6_port)); + } + + dst_node_get(st->node); + newst = kst_state_init(st->node, pe

[1/4] Distributed storage. Documentation.

2007-11-05 Thread Evgeniy Polyakov
dst_remove_alg() with given algorithm +pointer. diff --git a/Documentation/dst/dst.txt b/Documentation/dst/dst.txt new file mode 100644 index 000..a6ea126 --- /dev/null +++ b/Documentation/dst/dst.txt @@ -0,0 +1,69 @@ +Distributed storage. Design and implementation. +http://tservice.net.ru/~s0mbre/

[0/4] Distributed storage. Squizzed black-out of the dancing back-aching hippo.

2007-11-05 Thread Evgeniy Polyakov
Hi. I'm pleased to announce 7'th and the final release of the distributed storage subsystem (DST). It allows to form a storage on top of local and remote nodes and combine them in linear or mirroring setup, which in turn can be exported to remote nodes. Short changelog: * added stron

Re: [0/3] Distributed storage. Mirror algo extension for automatic recovery.

2007-10-27 Thread Evgeniy Polyakov
Hi Andrew. On Fri, Oct 26, 2007 at 01:55:38PM -0700, Andrew Morton ([EMAIL PROTECTED]) wrote: > > I'm pleased to announce sixth release of the distributed storage > > subsystem, which allows to form a storage on top of remote and local > > nodes, which in turn can be expo

Re: [0/3] Distributed storage. Mirror algo extension for automatic recovery.

2007-10-26 Thread Andrew Morton
On Thu, 18 Oct 2007 23:17:41 +0400 Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > I'm pleased to announce sixth release of the distributed storage > subsystem, which allows to form a storage on top of remote and local > nodes, which in turn can be exported to another storage

Re: Distributed storage. Move away from char device ioctls.

2007-10-26 Thread Evgeniy Polyakov
Returning back to this, since block based storage, which can act as a shared storage/transport layer, is ready with 5'th release of the DST. My couple of notes on proposed data distribution algorithm in FS. On Sun, Sep 16, 2007 at 03:07:11AM -0400, Kyle Moffett ([EMAIL PROTECTED]) wrote: > >I ac

[3/3] Distributed storage. Documentation and algorithms.

2007-10-18 Thread Evgeniy Polyakov
en algorithm +pointer. diff --git a/Documentation/dst/dst.txt b/Documentation/dst/dst.txt new file mode 100644 index 000..3b326aa --- /dev/null +++ b/Documentation/dst/dst.txt @@ -0,0 +1,66 @@ +Distributed storage. Design and implementation. +http://tservice.net.ru/~s0mbre/old/?sect

[2/3] Distributed storage. Network state machine.

2007-10-18 Thread Evgeniy Polyakov
struct sockaddr_in6 *sin = (struct sockaddr_in6 *)&addr; + printk(KERN_INFO "%s: Client: " + "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x:%d", + __func__, + NIP6(sin->sin6_addr), ntohs(sin-

[1/3] Distributed storage. Core files.

2007-10-18 Thread Evgeniy Polyakov
istate "Distributed storage" + depends on NET + select CONNECTOR + ---help--- + This driver allows to create a distributed storage. + +config DST_ALG_LINEAR + tristate "Linear distribution algorithm" + depends on DST + ---help--- + This modu

[0/3] Distributed storage. Mirror algo extension for automatic recovery.

2007-10-18 Thread Evgeniy Polyakov
Hi. I'm pleased to announce sixth release of the distributed storage subsystem, which allows to form a storage on top of remote and local nodes, which in turn can be exported to another storage as a node to form tree-like storages. This release includes mirroring algorithm extension,

Re: Distributed storage. Security attributes and ducumentation update.

2007-09-22 Thread Evgeniy Polyakov
Hi Pavel. On Mon, Sep 17, 2007 at 06:22:30PM +, Pavel Machek ([EMAIL PROTECTED]) wrote: > > I'm pleased to announce third release of the distributed storage > > subsystem, which allows to form a storage on top of remote and local > > nodes, which in turn can be exporte

Re: Distributed storage. Security attributes and ducumentation update.

2007-09-21 Thread Pavel Machek
Hi! > I'm pleased to announce third release of the distributed storage > subsystem, which allows to form a storage on top of remote and local > nodes, which in turn can be exported to another storage as a node to > form tree-like storages. How is this different from raid0/1 ove

Re: Distributed storage. Move away from char device ioctls.

2007-09-16 Thread Evgeniy Polyakov
On Sat, Sep 15, 2007 at 11:24:46AM -0600, Andreas Dilger ([EMAIL PROTECTED]) wrote: > > When Chris Mason announced btrfs, I found that quite a few new ideas > > are already implemented there, so I postponed project (although > > direction of the developement of the btrfs seems to move to the zfs

Re: Distributed storage. Move away from char device ioctls.

2007-09-16 Thread Kyle Moffett
On Sep 15, 2007, at 13:24:46, Andreas Dilger wrote: On Sep 15, 2007 16:29 +0400, Evgeniy Polyakov wrote: Yes, block device itself is not able to scale well, but it is the place for redundancy, since filesystem will just fail if underlying device does not work correctly and FS actually does n

Re: Distributed storage. Move away from char device ioctls.

2007-09-15 Thread Andreas Dilger
On Sep 15, 2007 12:20 -0400, Robin Humble wrote: > On Sat, Sep 15, 2007 at 10:35:16AM -0400, Jeff Garzik wrote: > >Lustre is tilted far too much towards high-priced storage, > > many (most?) Lustre deployments are with SATA and md raid5 and GigE - > can't get much cheaper than that. I have to ag

Re: Distributed storage. Move away from char device ioctls.

2007-09-15 Thread Andreas Dilger
On Sep 15, 2007 16:29 +0400, Evgeniy Polyakov wrote: > Yes, block device itself is not able to scale well, but it is the place > for redundancy, since filesystem will just fail if underlying device > does not work correctly and FS actually does not know about where it > should place redundancy bit

Re: Distributed storage. Move away from char device ioctls.

2007-09-15 Thread Robin Humble
On Sat, Sep 15, 2007 at 10:35:16AM -0400, Jeff Garzik wrote: >Robin Humble wrote: >>On Fri, Sep 14, 2007 at 03:07:46PM -0400, Jeff Garzik wrote: >>>I've been waiting for years for a smart person to come along and write a >>>POSIX-only distributed filesystem. >>it's called Lustre. >>works well, sca

Re: Distributed storage. Move away from char device ioctls.

2007-09-15 Thread Jeff Garzik
Robin Humble wrote: On Fri, Sep 14, 2007 at 03:07:46PM -0400, Jeff Garzik wrote: It is my hope that you will put your skills towards a distributed filesystem :) Of the current solutions, GFS (currently in kernel) scales poorly, and NFS v4.1 is amazingly bloated and overly complex. I've been

Re: Distributed storage. Move away from char device ioctls.

2007-09-15 Thread Robin Humble
On Fri, Sep 14, 2007 at 03:07:46PM -0400, Jeff Garzik wrote: >It is my hope that you will put your skills towards a distributed >filesystem :) Of the current solutions, GFS (currently in kernel) >scales poorly, and NFS v4.1 is amazingly bloated and overly complex. > >I've been waiting for years

Re: Distributed storage. Move away from char device ioctls.

2007-09-15 Thread Evgeniy Polyakov
Hi Mike. On Fri, Sep 14, 2007 at 10:54:56PM -0400, Mike Snitzer ([EMAIL PROTECTED]) wrote: > This distributed storage is very much needed; even if it were to act > as a more capable/performant replacement for NBD (or MD+NBD) in the > near term. Many high availability applications don

Re: Distributed storage. Move away from char device ioctls.

2007-09-15 Thread Evgeniy Polyakov
Hi Jeff. On Fri, Sep 14, 2007 at 03:07:46PM -0400, Jeff Garzik ([EMAIL PROTECTED]) wrote: > >Further TODO list includes: > >* implement optional saving of mirroring/linear information on the remote > > nodes (simple) > >* new redundancy algorithm (complex) > >* some thoughts about distributed

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread J. Bruce Fields
On Sat, Sep 15, 2007 at 12:08:42AM -0400, Jeff Garzik wrote: > J. Bruce Fields wrote: >> No, servers are required to support ordinary nfs operations to the >> metadata server. >> At least, that's the way it was last I heard, which was a while ago. I >> agree that it'd stink (for any number of reas

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread Jeff Garzik
J. Bruce Fields wrote: On Fri, Sep 14, 2007 at 06:32:11PM -0400, Jeff Garzik wrote: J. Bruce Fields wrote: On Fri, Sep 14, 2007 at 05:14:53PM -0400, Jeff Garzik wrote: NFSv4.1 adds to the fun, by throwing interoperability completely out the window. What parts are you worried about in partic

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread Mike Snitzer
On 9/14/07, Jeff Garzik <[EMAIL PROTECTED]> wrote: > Evgeniy Polyakov wrote: > > Hi. > > > > I'm pleased to announce fourth release of the distributed storage > > subsystem, which allows to form a storage on top of remote and local > > nodes, which in

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread J. Bruce Fields
On Fri, Sep 14, 2007 at 06:32:11PM -0400, Jeff Garzik wrote: > J. Bruce Fields wrote: >> On Fri, Sep 14, 2007 at 05:14:53PM -0400, Jeff Garzik wrote: >>> J. Bruce Fields wrote: On Fri, Sep 14, 2007 at 03:07:46PM -0400, Jeff Garzik wrote: > I've been waiting for years for a smart person to

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread Jeff Garzik
J. Bruce Fields wrote: On Fri, Sep 14, 2007 at 05:14:53PM -0400, Jeff Garzik wrote: J. Bruce Fields wrote: On Fri, Sep 14, 2007 at 03:07:46PM -0400, Jeff Garzik wrote: I've been waiting for years for a smart person to come along and write a POSIX-only distributed filesystem. What exactly do y

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread J. Bruce Fields
On Fri, Sep 14, 2007 at 05:14:53PM -0400, Jeff Garzik wrote: > J. Bruce Fields wrote: >> On Fri, Sep 14, 2007 at 03:07:46PM -0400, Jeff Garzik wrote: >>> I've been waiting for years for a smart person to come along and write a >>> POSIX-only distributed filesystem. > >> What exactly do you mean by

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread Jeff Garzik
J. Bruce Fields wrote: On Fri, Sep 14, 2007 at 03:07:46PM -0400, Jeff Garzik wrote: I've been waiting for years for a smart person to come along and write a POSIX-only distributed filesystem. What exactly do you mean by "POSIX-only"? Don't bother supporting attributes, file modes, and other

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread J. Bruce Fields
On Fri, Sep 14, 2007 at 03:07:46PM -0400, Jeff Garzik wrote: > My thoughts. But first a disclaimer: Perhaps you will recall me as one > of the people who really reads all your patches, and examines your code and > proposals closely. So, with that in mind... > > I question the value of distrib

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread Al Boldi
Jeff Garzik wrote: > Evgeniy Polyakov wrote: > > Hi. > > > > I'm pleased to announce fourth release of the distributed storage > > subsystem, which allows to form a storage on top of remote and local > > nodes, which in turn can be exported to another s

Re: Distributed storage. Move away from char device ioctls.

2007-09-14 Thread Jeff Garzik
Evgeniy Polyakov wrote: Hi. I'm pleased to announce fourth release of the distributed storage subsystem, which allows to form a storage on top of remote and local nodes, which in turn can be exported to another storage as a node to form tree-like storages. This release include

Re: Distributed storage. Security attributes and ducumentation update.

2007-09-13 Thread Paul E. McKenney
On Thu, Sep 13, 2007 at 04:22:59PM +0400, Evgeniy Polyakov wrote: > Hi Paul. > > On Mon, Sep 10, 2007 at 03:14:45PM -0700, Paul E. McKenney ([EMAIL > PROTECTED]) wrote: > > > Further TODO list includes: > > > * implement optional saving of mirroring/linear information on the remote > > > nodes

Re: Distributed storage. Security attributes and ducumentation update.

2007-09-13 Thread Evgeniy Polyakov
Hi Paul. On Mon, Sep 10, 2007 at 03:14:45PM -0700, Paul E. McKenney ([EMAIL PROTECTED]) wrote: > > Further TODO list includes: > > * implement optional saving of mirroring/linear information on the remote > > nodes (simple) > > * implement netlink based setup (simple) > > * new redundancy alg

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-09-01 Thread Daniel Phillips
On Friday 31 August 2007 14:41, Alasdair G Kergon wrote: > On Thu, Aug 30, 2007 at 04:20:35PM -0700, Daniel Phillips wrote: > > Resubmitting a bio or submitting a dependent bio from > > inside a block driver does not need to be throttled because all > > resources required to guarantee completion mu

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-08-31 Thread Alasdair G Kergon
On Thu, Aug 30, 2007 at 04:20:35PM -0700, Daniel Phillips wrote: > Resubmitting a bio or submitting a dependent bio from > inside a block driver does not need to be throttled because all > resources required to guarantee completion must have been obtained > _before_ the bio was allowed to procee

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-08-31 Thread Evgeniy Polyakov
Hi Daniel. On Thu, Aug 30, 2007 at 04:20:35PM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > On Wednesday 29 August 2007 01:53, Evgeniy Polyakov wrote: > > Then, if of course you will want, which I doubt, you can reread > > previous mails and find that it was pointed to that race and > > pos

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-08-30 Thread Daniel Phillips
On Wednesday 29 August 2007 01:53, Evgeniy Polyakov wrote: > Then, if of course you will want, which I doubt, you can reread > previous mails and find that it was pointed to that race and > possibilities to solve it way too long ago. What still bothers me about your response is that, while you kno

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-08-29 Thread Evgeniy Polyakov
On Tue, Aug 28, 2007 at 02:08:04PM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > On Tuesday 28 August 2007 10:54, Evgeniy Polyakov wrote: > > On Tue, Aug 28, 2007 at 10:27:59AM -0700, Daniel Phillips ([EMAIL > > PROTECTED]) wrote: > > > > We do not care about one cpu being able to increase

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-08-28 Thread Daniel Phillips
On Tuesday 28 August 2007 10:54, Evgeniy Polyakov wrote: > On Tue, Aug 28, 2007 at 10:27:59AM -0700, Daniel Phillips ([EMAIL PROTECTED]) > wrote: > > > We do not care about one cpu being able to increase its counter > > > higher than the limit, such inaccuracy (maximum bios in flight > > > thus ca

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-08-28 Thread Evgeniy Polyakov
On Tue, Aug 28, 2007 at 10:27:59AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > > We do not care about one cpu being able to increase its counter > > higher than the limit, such inaccuracy (maximum bios in flight thus > > can be more than limit, difference is equal to the number of CPUs - >

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-08-28 Thread Daniel Phillips
On Tuesday 28 August 2007 02:35, Evgeniy Polyakov wrote: > On Mon, Aug 27, 2007 at 02:57:37PM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > > Say Evgeniy, something I was curious about but forgot to ask you > > earlier... > > > > On Wednesday 08 August 2007 03:17, Evgeniy Polyakov wrote: > >

Re: Distributed storage.

2007-08-28 Thread Evgeniy Polyakov
On Fri, Aug 03, 2007 at 09:04:51AM +0400, Manu Abraham ([EMAIL PROTECTED]) wrote: > On 7/31/07, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > TODO list currently includes following main items: > > * redundancy algorithm (drop me a request of your own, but it is highly > > unlikley

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-08-28 Thread Evgeniy Polyakov
On Mon, Aug 27, 2007 at 02:57:37PM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > Say Evgeniy, something I was curious about but forgot to ask you > earlier... > > On Wednesday 08 August 2007 03:17, Evgeniy Polyakov wrote: > > ...All oerations are not atomic, since we do not care about prec

Re: [1/1] Block device throttling [Re: Distributed storage.]

2007-08-27 Thread Daniel Phillips
Say Evgeniy, something I was curious about but forgot to ask you earlier... On Wednesday 08 August 2007 03:17, Evgeniy Polyakov wrote: > ...All oerations are not atomic, since we do not care about precise > number of bios, but a fact, that we are close or close enough to the > limit. > ... in bi

Re: Distributed storage. Mirroring to any number of devices.

2007-08-14 Thread Evgeniy Polyakov
On Tue, Aug 14, 2007 at 07:20:49PM +0200, Jan Engelhardt ([EMAIL PROTECTED]) wrote: > >I'm pleased to announce second release of the distributed storage > >subsystem, which allows to form a storage on top of remote and local > >nodes, which in turn can be exported to anoth

Re: Distributed storage. Mirroring to any number of devices.

2007-08-14 Thread Jan Engelhardt
On Aug 14 2007 20:29, Evgeniy Polyakov wrote: > >I'm pleased to announce second release of the distributed storage >subsystem, which allows to form a storage on top of remote and local >nodes, which in turn can be exported to another storage as a node to >form tree-like stor

Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Daniel Phillips
On Tuesday 14 August 2007 05:46, Evgeniy Polyakov wrote: > > The throttling of the virtual device must begin in > > generic_make_request and last to ->endio. You release the throttle > > of the virtual device at the point you remap the bio to an > > underlying device, which you have convinced your

Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Evgeniy Polyakov
On Tue, Aug 14, 2007 at 05:32:29AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > On Tuesday 14 August 2007 04:50, Evgeniy Polyakov wrote: > > On Tue, Aug 14, 2007 at 04:35:43AM -0700, Daniel Phillips > ([EMAIL PROTECTED]) wrote: > > > On Tuesday 14 August 2007 04:30, Evgeniy Polyakov wrote:

  1   2   >