Re: [gentoo-user] Re: File system testing

2014-09-19 Thread Kerin Millar
On 18/09/2014 14:12, Alec Ten Harmsel wrote: On 09/18/2014 05:17 AM, Kerin Millar wrote: On 17/09/2014 21:20, Alec Ten Harmsel wrote: As far as HDFS goes, I would only set that up if you will use it for Hadoop or related tools. It's highly specific, and the performance is not good unless you'r

Re: [gentoo-user] Re: File system testing

2014-09-19 Thread J. Roeleveld
On Friday, September 19, 2014 10:56:59 AM Rich Freeman wrote: > On Fri, Sep 19, 2014 at 9:41 AM, James wrote: > > I think btrfs has tremendous potential. I tried ZFS a few times, > > but the installs are not part of gentoo, so they got borked > > uEFI, grubs to uuids, etc etc also were in the mix

Re: [gentoo-user] Re: File system testing

2014-09-19 Thread J. Roeleveld
On Friday, September 19, 2014 01:41:26 PM James wrote: > J. Roeleveld antarean.org> writes: > > Out of curiosity, what do you want to simulate? > > subsurface flows in porous medium. AKA carbon sequestration > by injection wells. You know, provide proof that those > that remove hydrocarbons and

Re: [gentoo-user] Re: File system testing

2014-09-19 Thread Rich Freeman
On Fri, Sep 19, 2014 at 9:41 AM, James wrote: > > I think btrfs has tremendous potential. I tried ZFS a few times, > but the installs are not part of gentoo, so they got borked > uEFI, grubs to uuids, etc etc also were in the mix. That was almost > a year ago. For what ever reason the clustering f

[gentoo-user] Re: File system testing

2014-09-19 Thread James
J. Roeleveld antarean.org> writes: > Out of curiosity, what do you want to simulate? subsurface flows in porous medium. AKA carbon sequestration by injection wells. You know, provide proof that those that remove hydrocarbons and actuall put the CO2 back and significantly mitigate the effects of

[gentoo-user] Re: File system testing

2014-09-18 Thread James
Hervé Guillemet guillemet.org> writes: > > Le 16/09/2014 21:07, James a écrit : > > > > By now many are familiar with my keen interest in clustering gentoo > > systems. So, what most cluster technologies use is a distributed file > > system on top of the local (HD/SDD) file system. > Have you

Re: [gentoo-user] Re: File system testing

2014-09-18 Thread Alec Ten Harmsel
On 09/18/2014 05:17 AM, Kerin Millar wrote: > On 17/09/2014 21:20, Alec Ten Harmsel wrote: >> As far as HDFS goes, I would only set that up if you will use it for >> Hadoop or related tools. It's highly specific, and the performance is >> not good unless you're doing a massively parallel read (wha

Re: [gentoo-user] Re: File system testing

2014-09-18 Thread J. Roeleveld
On Thursday, September 18, 2014 05:48:58 AM Rich Freeman wrote: > The HTML...it hurts my eyes... :) Apologies. > > My current understanding is: > > > > - ZFS is production ready, but due to licensing issues, not included in > > the > > kernel > > > > - BTRFS is included, but not yet productio

Re: [gentoo-user] Re: File system testing

2014-09-18 Thread Rich Freeman
The HTML...it hurts my eyes... :) On Thu, Sep 18, 2014 at 4:24 AM, J. Roeleveld wrote: > > On Wednesday, September 17, 2014 08:56:28 PM James wrote: > >> Alec Ten Harmsel alectenharmsel.com> writes: > >> > As far as HDFS goes, I would only set that up if you will use it for >> > Hadoop or relat

Re: [gentoo-user] Re: File system testing

2014-09-18 Thread Kerin Millar
On 17/09/2014 21:20, Alec Ten Harmsel wrote: As far as HDFS goes, I would only set that up if you will use it for Hadoop or related tools. It's highly specific, and the performance is not good unless you're doing a massively parallel read (what it was designed for). I can elaborate why if anyone

Re: [gentoo-user] Re: File system testing

2014-09-18 Thread J. Roeleveld
On Wednesday, September 17, 2014 08:56:28 PM James wrote: > Alec Ten Harmsel alectenharmsel.com> writes: > > As far as HDFS goes, I would only set that up if you will use it for > > Hadoop or related tools. It's highly specific, and the performance is > > not good unless you're doing a massively

Re: [gentoo-user] Re: File system testing

2014-09-18 Thread J. Roeleveld
On Wednesday, September 17, 2014 04:20:24 PM Alec Ten Harmsel wrote: > As far as HDFS goes, I would only set that up if you will use it for > Hadoop or related tools. It's highly specific, and the performance is > not good unless you're doing a massively parallel read (what it was > designed for).

Re: [gentoo-user] Re: File system testing

2014-09-18 Thread J. Roeleveld
On Wednesday, September 17, 2014 09:05:09 PM James wrote: > J. Roeleveld antarean.org> writes: > > AFS has caching and can survive temporary disappearance of the server. > > Excellent for low bandwidth connections. Most DFS have mechanisms to > deal with transient failures, but not as generaous

[gentoo-user] Re: File system testing

2014-09-17 Thread James
J. Roeleveld antarean.org> writes: > AFS has caching and can survive temporary disappearance of the server. Excellent for low bandwidth connections. Most DFS have mechanisms to deal with transient failures, but not as generaous on the time-scale as AFS. I believe, if I recall correctly, these h

[gentoo-user] Re: File system testing

2014-09-17 Thread James
Alec Ten Harmsel alectenharmsel.com> writes: > As far as HDFS goes, I would only set that up if you will use it for > Hadoop or related tools. It's highly specific, and the performance is > not good unless you're doing a massively parallel read (what it was > designed for). I can elaborate why i

Re: [gentoo-user] Re: File system testing

2014-09-17 Thread Alec Ten Harmsel
As far as HDFS goes, I would only set that up if you will use it for Hadoop or related tools. It's highly specific, and the performance is not good unless you're doing a massively parallel read (what it was designed for). I can elaborate why if anyone is actually interested. We use Lustre for our

Re: [gentoo-user] Re: File system testing

2014-09-17 Thread J. Roeleveld
On Wednesday, September 17, 2014 03:55:56 PM James wrote: > J. Roeleveld antarean.org> writes: > > > Distributed File Systems (DFS): > > > > > Local (Device) File Systems LFS: > > Is my understanding correct that the top list all require one of > > the bottom list? > > Eg. the "clustering" FSs

[gentoo-user] Re: File system testing

2014-09-17 Thread James
J. Roeleveld antarean.org> writes: > > Distributed File Systems (DFS): > > Local (Device) File Systems LFS: > Is my understanding correct that the top list all require one of > the bottom list? > Eg. the "clustering" FSs only ensure the files on the LFSs are > duplicated/spread over the var