Re: Storage server

2012-09-17 Thread Veljko
On Thu, Sep 13, 2012 at 06:24:45PM -0500, Stan Hoeppner wrote: > Due to its allocation group design, continually growing an XFS > filesystem in such small increments, with this metadata heavy backup > workload, will yield very poor performance. Additionally, putting an > XFS filesystem atop an LV

Re: Storage server

2012-09-17 Thread Veljko
On Fri, Sep 14, 2012 at 10:48:54AM +0200, Denis Witt wrote: > I'm currently testing obnam on our external Backup-Server together with > 6 clients. It's very easy to set up. Restore could be nicer if you need > an older version of some file but it's rather fast and it is possible > to restore single

Re: Storage server

2012-09-16 Thread Martin Steigerwald
Am Sonntag, 16. September 2012 schrieb Stan Hoeppner: > On 9/16/2012 7:38 AM, Martin Steigerwald wrote: > > I have always recommended to leave at least 10-15% free, but from a > > discussion on XFS mailinglist where you took part, I learned that > > depending on use case for large volumes even more

Re: Storage server

2012-09-16 Thread Stan Hoeppner
On 9/16/2012 7:38 AM, Martin Steigerwald wrote: > I have always recommended to leave at least 10-15% free, but from a > discussion on XFS mailinglist where you took part, I learned that > depending on use case for large volumes even more free space might be > necessary for performant long term

Re: Storage server

2012-09-16 Thread Martin Steigerwald
Hi Kelly, Am Samstag, 15. September 2012 schrieb Kelly Clowers: > On Fri, Sep 14, 2012 at 2:51 PM, Stan Hoeppner wrote: > > On 9/14/2012 11:29 AM, Kelly Clowers wrote: > >> On Thu, Sep 13, 2012 at 4:45 PM, Stan Hoeppner > >> wrote: > >>> On 9/13/2012 5:20 AM, Veljko wrote: > On Tue, Sep 11

Re: Storage server

2012-09-16 Thread Martin Steigerwald
Am Freitag, 14. September 2012 schrieb Stan Hoeppner: > On 9/14/2012 7:57 AM, Martin Steigerwald wrote: > > Am Freitag, 14. September 2012 schrieb Stan Hoeppner: > >> Thus my advice to you is: > >> > >> Do not use LVM. Directly format the RAID10 device using the > >> mkfs.xfs defaults. mkfs.xfs

Re: Storage server

2012-09-16 Thread Martin Steigerwald
Am Samstag, 15. September 2012 schrieb Bob Proulx: > Martin Steigerwald wrote: > > Am Freitag, 7. September 2012 schrieb Bob Proulx: > > > Unfortunately I have some recent FUD concerning xfs. I have had > > > some recent small idle xfs filesystems trigger kernel watchdog > > > timer ... > > > due

Re: Storage server

2012-09-15 Thread Bob Proulx
Stan Hoeppner wrote: > Bob Proulx wrote: > > But in the future when when Debian Jessie is being released I am going > > to be reading then on the mailing list about how old and bad Linux 3.2 > > is and how it should not be used because it is too old. > > So what you're saying here is that Jessie s

Re: Storage server

2012-09-15 Thread Stan Hoeppner
On 9/15/2012 3:36 AM, Bob Proulx wrote: > But in the future when when Debian Jessie is being released I am going > to be reading then on the mailing list about how old and bad Linux 3.2 > is and how it should not be used because it is too old. So what you're saying here is that Jessie should be r

Re: Storage server

2012-09-15 Thread Kelly Clowers
On Sat, Sep 15, 2012 at 1:36 AM, Bob Proulx wrote: > > Meanwhile I am running Sid on my main desktop machine. I upgrade it > daily. I report bugs as I find them. I am doing so specifically so I > can test and find and report bugs. Wow, impressive. I run unstable+experimental, but I think I hav

Re: Storage server

2012-09-15 Thread Bob Proulx
Martin Steigerwald wrote: > Am Freitag, 7. September 2012 schrieb Bob Proulx: > > Unfortunately I have some recent FUD concerning xfs. I have had some > > recent small idle xfs filesystems trigger kernel watchdog timer > > ... > > due to these lockups. Squeeze. Everything current. But when idle

Re: Storage server [solved???]

2012-09-14 Thread Paul E Condon
'Solved' is not a proper description. Better to say that I have discovered some serious misunderstanding on my part. It would be a serious waste of other peoples time to extend this sub-thread with a detailed explanation. Sorry. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUB

Re: Storage server

2012-09-14 Thread Kelly Clowers
On Fri, Sep 14, 2012 at 2:51 PM, Stan Hoeppner wrote: > On 9/14/2012 11:29 AM, Kelly Clowers wrote: >> On Thu, Sep 13, 2012 at 4:45 PM, Stan Hoeppner >> wrote: >>> On 9/13/2012 5:20 AM, Veljko wrote: On Tue, Sep 11, 2012 at 08:34:51AM -0500, Stan Hoeppner wrote: > One of the big reasons

Re: Storage server

2012-09-14 Thread Stan Hoeppner
On 9/14/2012 11:29 AM, Kelly Clowers wrote: > On Thu, Sep 13, 2012 at 4:45 PM, Stan Hoeppner wrote: >> On 9/13/2012 5:20 AM, Veljko wrote: >>> On Tue, Sep 11, 2012 at 08:34:51AM -0500, Stan Hoeppner wrote: One of the big reasons (other than cost) that I mentioned this card is that Adapte

Re: Storage server

2012-09-14 Thread Stan Hoeppner
On 9/14/2012 7:57 AM, Martin Steigerwald wrote: > Am Freitag, 14. September 2012 schrieb Stan Hoeppner: >> Thus my advice to you is: >> >> Do not use LVM. Directly format the RAID10 device using the mkfs.xfs >> defaults. mkfs.xfs will read the md configuration and automatically >> align the files

Re: Storage server

2012-09-14 Thread Paul E Condon
On 20120910_053746, Stan Hoeppner wrote: > On 9/9/2012 3:25 PM, Paul E Condon wrote: > > > I've been following this thread from its beginning. My initial reading > > of OP's post was to marvel at the thought that so many things/tasks > > could be done with a single box in a single geek's cubicle.

Re: Storage server

2012-09-14 Thread Kelly Clowers
On Thu, Sep 13, 2012 at 4:45 PM, Stan Hoeppner wrote: > On 9/13/2012 5:20 AM, Veljko wrote: >> On Tue, Sep 11, 2012 at 08:34:51AM -0500, Stan Hoeppner wrote: >>> One of the big reasons (other than cost) that I mentioned this card is >>> that Adaptec tends to be more forgiving with non RAID specifi

Re: Storage server

2012-09-14 Thread Martin Steigerwald
Am Freitag, 14. September 2012 schrieb Stan Hoeppner: > Thus my advice to you is: > > Do not use LVM. Directly format the RAID10 device using the mkfs.xfs > defaults. mkfs.xfs will read the md configuration and automatically > align the filesystem to the stripe width. Just for completeness: It

Re: Storage server

2012-09-14 Thread Stan Hoeppner
On 9/14/2012 4:48 AM, Pertti Kosunen wrote: > On 14.9.2012 2:45, Stan Hoeppner wrote: >> Consumer hard drives will not work with most RAID cards. As a general >> rule, RAID cards require enterprise SATA drives or SAS drives. > > http://wdc.com/en/products/products.aspx?id=810 > http://www.anandte

Re: Storage server

2012-09-14 Thread Jon Dowland
On Thu, Sep 13, 2012 at 12:20:55PM +0200, Veljko wrote: > Can you please explain what design flaw is that? Isn't directory with > complete backup (but not occupying that much space due to hard links > usage) very usable for backup? If slow work can be avoided by the use of > XFS, what would be wron

Re: Storage server

2012-09-14 Thread Pertti Kosunen
On 14.9.2012 2:45, Stan Hoeppner wrote: Consumer hard drives will not work with most RAID cards. As a general rule, RAID cards require enterprise SATA drives or SAS drives. http://wdc.com/en/products/products.aspx?id=810 http://www.anandtech.com/show/6157/ Western Digitals new Red series is R

Re: Storage server

2012-09-14 Thread Denis Witt
On Thu, 13 Sep 2012 12:22:45 +0200 Veljko wrote: > obnam and rdiff-backup seems to use less space, but I also like very > clear representation of backups on rsnapshot. But during few days of > testing each of them I'll know what to use. I think rdiff-backup is a good choice for your needs. It h

Re: Storage server

2012-09-14 Thread Denis Witt
On Thu, 13 Sep 2012 12:21:44 +0200 Veljko wrote: > I've heard of it, but don't know anyone who uses it. Any experience > with it? Our former Hosting Provider used Amanda, I never liked it (but maybe because of the interface the Provider used for it). I think for Veljko needs it is much to comple

Re: Storage server

2012-09-13 Thread Stan Hoeppner
On 9/13/2012 5:20 AM, Veljko wrote: > On Tue, Sep 11, 2012 at 08:34:51AM -0500, Stan Hoeppner wrote: >> One of the big reasons (other than cost) that I mentioned this card is >> that Adaptec tends to be more forgiving with non RAID specific >> (ERC/TLER) drives, and lists your Seagate 3TB drives as

Re: Storage server

2012-09-13 Thread Stan Hoeppner
On 9/13/2012 5:21 AM, Veljko wrote: > On Tue, Sep 11, 2012 at 05:44:46PM -0500, Stan Hoeppner wrote: >> On 9/11/2012 10:29 AM, Jon Dowland wrote: >> >>> Actually, lots and lots of small files is the worst use-case for rsnapshot, >>> and >>> the reason I believe it should be avoided. It creates lar

Re: Storage server

2012-09-13 Thread lee
Veljko writes: > On Wed, Sep 12, 2012 at 06:49:22PM +0200, lee wrote: >> Denis Witt writes: >> >> > Anyway, I have some comparison data. I have a backup server that saves >> > data from 5 other server at our hosting company using rsnapshot. The >> > backups are kept for 14 days. >> > >> > rsnap

Re: Storage server

2012-09-13 Thread Veljko
On Thu, Sep 13, 2012 at 12:16:21PM +0100, Tony van der Hoff wrote: > When I used tape for backup, I used Amanda, and it did what it was > supposed to do very well, with tape contents indexes, and a media > rotation pattern. > > However, in a non-tape environment with only a few machines, I found i

Re: Storage server

2012-09-13 Thread Tony van der Hoff
On 13/09/12 11:21, Veljko wrote: > On Wed, Sep 12, 2012 at 06:49:22PM +0200, lee wrote: >> Denis Witt writes: >> >>> Anyway, I have some comparison data. I have a backup server that saves >>> data from 5 other server at our hosting company using rsnapshot. The >>> backups are kept for 14 days. >>>

Re: Storage server

2012-09-13 Thread Veljko
On Wed, Sep 12, 2012 at 10:46:37AM +0200, Denis Witt wrote: > On Tue, 11 Sep 2012 16:29:22 +0100 > Jon Dowland wrote: > > > Denis' answer is very good, I won't re-iterate his points. > > Thanks. And also thanks for pointing out the Hardlinks thing, I > over-read the "lots of small files" part in

Re: Storage server

2012-09-13 Thread Veljko
On Wed, Sep 12, 2012 at 06:49:22PM +0200, lee wrote: > Denis Witt writes: > > > Anyway, I have some comparison data. I have a backup server that saves > > data from 5 other server at our hosting company using rsnapshot. The > > backups are kept for 14 days. > > > > rsnapshot: > > bup: > > obnam:

Re: Storage server

2012-09-13 Thread Veljko
On Tue, Sep 11, 2012 at 05:44:46PM -0500, Stan Hoeppner wrote: > On 9/11/2012 10:29 AM, Jon Dowland wrote: > > > Actually, lots and lots of small files is the worst use-case for rsnapshot, > > and > > the reason I believe it should be avoided. It creates large hard-link trees > > and > > with lo

Re: Storage server

2012-09-13 Thread Veljko
On Wed, Sep 12, 2012 at 01:54:18PM +0100, Jon Dowland wrote: > On Mon, Sep 10, 2012 at 08:03:43PM +0300, Andrei POPESCU wrote: > > http://www.taobackup.com/ > > Yes indeed, great read. > > Also this: http://www.jwz.org/doc/backups.html > > A single external drive, normally stored away from the s

Re: Storage server

2012-09-13 Thread Veljko
On Wed, Sep 12, 2012 at 01:50:04PM +0100, Jon Dowland wrote: > On Tue, Sep 11, 2012 at 05:44:46PM -0500, Stan Hoeppner wrote: > > Which is why I recommend XFS. It is exceptionally fast at > > traversing large btrees. You'll need the 3.2 bpo kernel for > > Squeeze. The old as dirt 2.6.32 kernel d

Re: Storage server

2012-09-13 Thread Veljko
On Tue, Sep 11, 2012 at 08:34:51AM -0500, Stan Hoeppner wrote: > One of the big reasons (other than cost) that I mentioned this card is > that Adaptec tends to be more forgiving with non RAID specific > (ERC/TLER) drives, and lists your Seagate 3TB drives as compatible. LSI > and other controllers

Re: Storage server

2012-09-13 Thread Veljko
On Tue, Sep 11, 2012 at 04:04:16PM +0200, Ralf Mardorf wrote: > The cheapest, but anyway reliable German retailer for all kinds of > electronic gear: > http://www.reichelt.de/index.html?;ACTION=103;LA=2;MANUFACTURER=adaptec;SID=12UE9B@H8AAAIAAEcGSWU702e805c66e3a1b7cce75cd098027793 > Perhaps you'll

Re: Storage server

2012-09-12 Thread lee
Denis Witt writes: > Anyway, I have some comparison data. I have a backup server that saves > data from 5 other server at our hosting company using rsnapshot. The > backups are kept for 14 days. > > rsnapshot: > bup: > obnam: > rdiff-backup: How about amanda? It hasn't been mentioned yet and mig

Re: Storage server

2012-09-12 Thread Jon Dowland
On Mon, Sep 10, 2012 at 08:03:43PM +0300, Andrei POPESCU wrote: > http://www.taobackup.com/ Yes indeed, great read. Also this: http://www.jwz.org/doc/backups.html A single external drive, normally stored away from the server, would be enough to have a backup that would survive the host going up

Re: Storage server

2012-09-12 Thread Jon Dowland
On Tue, Sep 11, 2012 at 05:44:46PM -0500, Stan Hoeppner wrote: > Which is why I recommend XFS. It is exceptionally fast at traversing large > btrees. You'll need the 3.2 bpo kernel for Squeeze. The old as dirt 2.6.32 > kernel doesn't contain any of the recent (last 3 years) metadata > optimizati

Re: Storage server

2012-09-12 Thread Denis Witt
On Tue, 11 Sep 2012 16:29:22 +0100 Jon Dowland wrote: > Denis' answer is very good, I won't re-iterate his points. Thanks. And also thanks for pointing out the Hardlinks thing, I over-read the "lots of small files" part in Velkjos Mail. Anyway, I have some comparison data. I have a backup serve

Re: Storage server

2012-09-11 Thread Stan Hoeppner
On 9/11/2012 10:29 AM, Jon Dowland wrote: > Actually, lots and lots of small files is the worst use-case for rsnapshot, > and > the reason I believe it should be avoided. It creates large hard-link trees > and > with lots and lots of small files, the filesystem metadata for the trees can > consu

Re: Storage server

2012-09-11 Thread lee
Veljko writes: > On Mon, Sep 10, 2012 at 07:47:52PM +0200, lee wrote: >> Did you get it to actually install on the RAID and to boot from that? >> Last time I tried with a RAID-1, it didn't work. It's ridiculously >> difficult to get it set up so that everything is on software raid. > > Yes, every

Re: Storage server

2012-09-11 Thread Jon Dowland
Denis' answer is very good, I won't re-iterate his points. On Tue, Sep 11, 2012 at 01:26:48PM +0200, Veljko wrote: > Thanks for your valuable input. So, in case I have to backup lot of > small files and only some of them are changed I should go with > rsnapshot. If there are big text files that ch

Re: Storage server

2012-09-11 Thread Ralf Mardorf
On Tue, 2012-09-11 at 16:04 +0200, Ralf Mardorf wrote: > On Tue, 2012-09-11 at 08:34 -0500, Stan Hoeppner wrote: > > On 9/10/2012 10:41 AM, Veljko wrote: > > > On Mon, Sep 10, 2012 at 08:05:49AM -0500, Stan Hoeppner wrote: > > >>> I'm not able to find that card here (and I haven't so far), can I >

Re: Storage server

2012-09-11 Thread Ralf Mardorf
On Tue, 2012-09-11 at 08:34 -0500, Stan Hoeppner wrote: > On 9/10/2012 10:41 AM, Veljko wrote: > > On Mon, Sep 10, 2012 at 08:05:49AM -0500, Stan Hoeppner wrote: > >>> I'm not able to find that card here (and I haven't so far), can I > have another one? > >> > >> That's hard to believe given the wo

Re: Storage server

2012-09-11 Thread Stan Hoeppner
On 9/11/2012 6:26 AM, Veljko wrote: > Debian reserve 1MB on start of the partition, but I > guess that part is used for MBR. The MBR is stored entirely in the first sector of the drive and is only 512 bytes in size. It includes the bootstrap code, partition table, and boot signature. The reason

Re: Storage server

2012-09-11 Thread Stan Hoeppner
On 9/11/2012 4:43 AM, Jon Dowland wrote: > I would say that neither hardware nor software RAID are a replacement for > a working backup scheme. Absolutely correct. RAID protects against drive failure, period. It doesn't protect against accidental file deletion, overwriting a new file with an old

Re: Storage server

2012-09-11 Thread Stan Hoeppner
On 9/10/2012 10:41 AM, Veljko wrote: > On Mon, Sep 10, 2012 at 08:05:49AM -0500, Stan Hoeppner wrote: >>> I'm not able to find that card here (and I haven't so far), can I have >>> another one? >> >> That's hard to believe given the worldwide penetration Adaptec has, and >> the fact UPS/FedEx ship

Re: Storage server

2012-09-11 Thread Denis Witt
On Tue, 11 Sep 2012 13:26:48 +0200 Veljko wrote: > Would it be reasonable to use them both where appropriate or thats > just unnecessary complexity? Hi Veljko, I prefer backups as simple as it could get (one reason why I use rsnapshot). So personally I wouldn't mix. But if you may want to pr

Re: Storage server

2012-09-11 Thread Veljko
On Mon, Sep 10, 2012 at 08:06:16PM +0200, Martin Steigerwald wrote: > If you made sure to explain the risks to your boss you can say in case > anything bad happens: I recommended doing backup in a different, safer way > than you allowed me to do it and thats the result. Thats exactly what I had

Re: Storage server

2012-09-11 Thread Veljko
On Tue, Sep 11, 2012 at 09:45:14PM +1200, Chris Bannister wrote: > On Mon, Sep 10, 2012 at 05:38:10PM +0200, Veljko wrote: > > Not that hard to comprehend. My boss sees backup as necessary evil. And > > only after I pushed it. Before I got here, there was no backup. None > > whatsover. I was baffle

Re: Storage server

2012-09-11 Thread Veljko
On Tue, Sep 11, 2012 at 10:41:04AM +0200, Denis Witt wrote: > On Mon, 10 Sep 2012 17:38:22 +0200 > Veljko wrote: > > > Any particular reason for avoiding rsnapshot? What are advantages of > > using rdiff-backup or obnam? > > Hi Veljko, > > I don't know a reason why someone should avoid rsnapsho

Re: Storage server

2012-09-11 Thread Veljko
On Mon, Sep 10, 2012 at 07:47:52PM +0200, lee wrote: > Did you get it to actually install on the RAID and to boot from that? > Last time I tried with a RAID-1, it didn't work. It's ridiculously > difficult to get it set up so that everything is on software raid. Yes, everything is on RAID. 2 boot

Re: Storage server

2012-09-11 Thread Veljko
On Mon, Sep 10, 2012 at 08:16:00PM +0200, Martin Steigerwald wrote: > GRUB needs a space between MBR and first partition. Maybe that space was to > small? Or more likely you GPT partitioned the disk (as its 3 TB and MBR > does only work upto 2 TB)? Then you need a BIOS boot partition. Unless you

Re: Storage server

2012-09-11 Thread Veljko
On Mon, Sep 10, 2012 at 08:03:43PM +0300, Andrei POPESCU wrote: > If you ignore the references to the proprietary backup software this is > a very interesting reading > > http://www.taobackup.com/ > > Kind regards, > Andrei Yes, very interesting. Thanks Andrei! Regards, Veljko -- To UNSUBSC

Re: Storage server

2012-09-11 Thread Chris Bannister
On Mon, Sep 10, 2012 at 05:38:10PM +0200, Veljko wrote: > Not that hard to comprehend. My boss sees backup as necessary evil. And > only after I pushed it. Before I got here, there was no backup. None > whatsover. I was baffled. And I had situation few days on my arrival, > that one of databases go

Re: Storage server

2012-09-11 Thread Jon Dowland
I would say that neither hardware nor software RAID are a replacement for a working backup scheme. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120911094342.GA118

Re: Storage server

2012-09-11 Thread Denis Witt
On Mon, 10 Sep 2012 17:38:22 +0200 Veljko wrote: > Any particular reason for avoiding rsnapshot? What are advantages of > using rdiff-backup or obnam? Hi Veljko, I don't know a reason why someone should avoid rsnapshot. rdiff-backup is very similar to rsnapshot but handles the backup generation

Re: Storage server

2012-09-10 Thread lee
Veljko writes: > I didn't till 30 minutes ago. :) I just installed it for exercise if > nothing else. Had a problem with booting. > > "Unable to install GRUB in /dev/sda > Executing 'grub-intall /dev/sda' failed. > This is a fatal error." > > After creating 1MB partition at the beginning of ever

Re: Storage server

2012-09-10 Thread Martin Steigerwald
Am Montag, 10. September 2012 schrieb Veljko: > On Mon, Sep 10, 2012 at 08:05:49AM -0500, Stan Hoeppner wrote: […] > > > In case I don't get that card, > > > should I remove /boot from RAID1? > > > > Post the output of > > > > ~$ cat /proc/mdstat > > > > I was under the impression you didn't hav

Re: Storage server

2012-09-10 Thread Martin Steigerwald
Am Montag, 10. September 2012 schrieb Veljko: [… no backup before and then backup as necessary evil …] > > If so, if I would be in the position to say no, I would just say "no > > thanks, search yourself a different idiot for setting up such an > > insane setup". I understand, you probably do n

Re: Storage server

2012-09-10 Thread Andrei POPESCU
On Lu, 10 sep 12, 17:38:39, Veljko wrote: > > I've never thought that RAID is backup. It's not. Server I'm trying to > set up is backup. It's not perfect solution, but is better then nothing. > Yes, in a perfect world I would set another one in case something > happened to this one, but that's the

Re: Storage server

2012-09-10 Thread lee
Stan Hoeppner writes: > Linux RAID is great in the right hands when used for appropriate > workloads. Too many people are using it who should not be, and giving > it a bad rap due to no fault of the software. Hm, interesting, so what would you say we should use it for and for what not? I'm usin

Re: Storage server

2012-09-10 Thread Veljko
On Mon, Sep 10, 2012 at 08:05:49AM -0500, Stan Hoeppner wrote: > > I'm not able to find that card here (and I haven't so far), can I have > > another one? > > That's hard to believe given the worldwide penetration Adaptec has, and > the fact UPS/FedEx ship worldwide. What country are you in? I'

Re: Storage server

2012-09-10 Thread Veljko
On Mon, Sep 10, 2012 at 04:02:54PM +0200, Martin Steigerwald wrote: > > Like I said, it's several dedicated, mostly web servers with users > > uploaded content on one of them (that part is expected to grow). None > > of them is in the same data center. > > Okay, so thats fine. > > I would still n

Re: Storage server

2012-09-10 Thread Veljko
On Mon, Sep 10, 2012 at 02:07:47PM +0100, Jon Dowland wrote: > On Sat, Sep 08, 2012 at 06:49:45PM +0200, Veljko wrote: > > a) backup (backup server for several dedicated (mainly) web servers). > > It will contain incremental backups, so only first running will take a > > lot of time, rsnapsho

Re: Storage server

2012-09-10 Thread Veljko
On Sat, Sep 08, 2012 at 09:53:33PM +0200, Martin Steigerwald wrote: > For rsnapshot in my experience you need monitoring cause if it fails it > just complains to its log file and even just puts the rsync error code > without the actual error message there last I checked. > > Let monitoring chec

Re: Storage server

2012-09-10 Thread The Wanderer
On 09/10/2012 10:16 AM, Stan Hoeppner wrote: On 9/10/2012 8:19 AM, The Wanderer wrote: But from what I'm told, hardware RAID has the downside that it often relies on the exact model of RAID card; if the card dies, you'll need an exact duplicate in order to be able to mount the RAID. You've

Re: Storage server

2012-09-10 Thread Martin Steigerwald
Am Montag, 10. September 2012 schrieb Jon Dowland: > On Sat, Sep 08, 2012 at 09:51:05PM +0200, lee wrote: > > Some people have argued it's even better to use software raid than a > > hardware raid controller because software raid doesn't depend on > > particular controller cards that can fail and c

Re: Storage server

2012-09-10 Thread Stan Hoeppner
On 9/10/2012 8:19 AM, The Wanderer wrote: > But from what I'm told, hardware RAID has the downside that it often > relies on > the exact model of RAID card; if the card dies, you'll need an exact > duplicate > in order to be able to mount the RAID. You've been misinformed. And, given your admis

Re: Storage server

2012-09-10 Thread Martin Steigerwald
Am Montag, 10. September 2012 schrieb Veljko: > On Sat, Sep 08, 2012 at 09:28:09PM +0200, Martin Steigerwald wrote: > > Consider the consequenzes: > > > > If the server fails, you possibly wouldn´t know why cause the > > monitoring information wouldn´t be available anymore. So at least > > least N

Re: Storage server

2012-09-10 Thread Stan Hoeppner
On 9/10/2012 8:11 AM, Jon Dowland wrote: > On Sat, Sep 08, 2012 at 09:51:05PM +0200, lee wrote: >> Some people have argued it's even better to use software raid than a >> hardware raid controller because software raid doesn't depend on >> particular controller cards that can fail and can be difficu

Re: Storage server

2012-09-10 Thread The Wanderer
On 09/10/2012 09:05 AM, Stan Hoeppner wrote: On 9/10/2012 5:47 AM, Veljko wrote: There is something that is not clear to me. You recommended hardware RAID as superior solution. I already knew that it is the case, but I thought that linux software RAID is also some solution. You mean "same"

Re: Storage server

2012-09-10 Thread Jon Dowland
On Sat, Sep 08, 2012 at 09:51:05PM +0200, lee wrote: > Some people have argued it's even better to use software raid than a > hardware raid controller because software raid doesn't depend on > particular controller cards that can fail and can be difficult to > replace. Besides that, software raid i

Re: Storage server

2012-09-10 Thread Jon Dowland
On Sat, Sep 08, 2012 at 06:49:45PM +0200, Veljko wrote: > a) backup (backup server for several dedicated (mainly) web servers). > It will contain incremental backups, so only first running will take a > lot of time, rsnapshot Best avoid rsnapshot. Use (at least) rdiff-backup instead, which i

Re: Storage server

2012-09-10 Thread Stan Hoeppner
On 9/10/2012 5:47 AM, Veljko wrote: > Not all of us have that kind of luxury to be that picky about our job, > but I get your point. Small companies with really tight purse strings may seem fine this week, then suddenly fold next week, everyone loses their jobs in the process. >> Get yourself a

Re: Storage server

2012-09-10 Thread The Wanderer
On 09/09/2012 02:37 AM, Stan Hoeppner wrote: On 9/7/2012 3:16 PM, Bob Proulx wrote: Whjat? Are you talking crash recovery boot time "fsck"? With any modern journaled FS log recovery is instantaneous. If you're talking about an actual structure check, XFS is pretty quick regardless of inode

Re: Storage server

2012-09-10 Thread Veljko
On Sun, Sep 09, 2012 at 03:42:12AM -0500, Stan Hoeppner wrote: > Stop here. Never use a production system as a test rig. Noted. > You can build a complete brand new AMD dedicated test machine with parts > from Newegg for $238 USD, sans KB/mouse/monitor, which you already have. > Boot it up then

Re: Storage server

2012-09-10 Thread Veljko
On Sat, Sep 08, 2012 at 09:28:09PM +0200, Martin Steigerwald wrote: > Consider the consequenzes: > > If the server fails, you possibly wouldn´t know why cause the monitoring > information wouldn´t be available anymore. So at least least Nagios / > Icingo send out mails, in case these are not sto

Re: Storage server

2012-09-10 Thread Veljko
On Sat, Sep 08, 2012 at 09:59:35PM +0200, Martin Steigerwald wrote: > Could it be that you intend to provide hosted monitoring, backup and > fileservices for an customer and while at it use the same machine for > testing own stuff? > > If so: Don´t. > > Thats at least my advice. (In addition to

Re: Storage server

2012-09-10 Thread Stan Hoeppner
On 9/9/2012 3:25 PM, Paul E Condon wrote: > I've been following this thread from its beginning. My initial reading > of OP's post was to marvel at the thought that so many things/tasks > could be done with a single box in a single geek's cubicle. One consumer quad core AMD Linux box of today can

Re: Storage server

2012-09-09 Thread Paul E Condon
On 20120909_040911, Stan Hoeppner wrote: > On 9/8/2012 2:53 PM, Martin Steigerwald wrote: > > > I would love to learn more about those really big XFS installations and > > how there were made. I never dealt with more than about 4 TiB big XFS > > setups. > > About the only information that's sti

Re: Storage server

2012-09-09 Thread Stan Hoeppner
On 9/8/2012 2:53 PM, Martin Steigerwald wrote: > I would love to learn more about those really big XFS installations and > how there were made. I never dealt with more than about 4 TiB big XFS > setups. About the only information that's still available is at the link below, and it lacks configu

Re: Storage server

2012-09-09 Thread Stan Hoeppner
On 9/8/2012 1:10 PM, Martin Steigerwald wrote: > Am Freitag, 7. September 2012 schrieb Stan Hoeppner: >> On 9/7/2012 12:42 PM, Dan Ritter wrote: > […] >>> Now, the next thing: I know it's tempting to make a single >>> filesystem over all these disks. Don't. The fsck times will be >>> horrendous. Ma

Re: Storage server

2012-09-09 Thread Stan Hoeppner
On 9/8/2012 11:49 AM, Veljko wrote: > Well, it did sound a little to complex and that is why I posted to this > list, hoping to hear some other opinions. > > 1. This machine will be used for > a) backup (backup server for several dedicated (mainly) web servers). > It will contain incremental

Re: Storage server

2012-09-08 Thread Stan Hoeppner
On 9/7/2012 3:16 PM, Bob Proulx wrote: > Agreed. But for me it isn't about the fsck time. It is about the > size of the problem. If you have full 100G filesystem and there is a > problem then you have a 100G problem. It is painful. But you can > handle it. If you have a full 10T filesystem a

Re: Storage server

2012-09-08 Thread lee
Veljko writes: > On Fri, Sep 07, 2012 at 09:43:57PM -0400, tdowg1 news wrote: > > If it was my call, I would go with high-end RAID card as well. But in > this case I have to work without them. However, I've heard that > software RAID is good for one thing. You can rebuild it in any other > machin

Re: Storage server

2012-09-08 Thread Martin Steigerwald
Am Samstag, 8. September 2012 schrieb Veljko: > Well, it did sound a little to complex and that is why I posted to this > list, hoping to hear some other opinions. > > 1. This machine will be used for > a) backup (backup server for several dedicated (mainly) web servers). > It will contain in

Re: Storage server

2012-09-08 Thread Martin Steigerwald
Am Samstag, 8. September 2012 schrieb Martin Steigerwald: > > And, of course, thanks for your time and valuable advices, Stan, I've > > read some of your previous posts on this list and know you're storage > > guru. > > It wasn´t Stan who wrote the mail you replied to here, but yes I think > I ca

Re: Storage server

2012-09-08 Thread Martin Steigerwald
Am Samstag, 8. September 2012 schrieb Veljko: > On Fri, Sep 07, 2012 at 01:26:13PM -0500, Stan Hoeppner wrote: > > On 9/7/2012 11:29 AM, Veljko wrote: > > > I'm in the process of making new backup server, so I'm thinking of > > > best way of doing it. I have 4 3TB disks and I'm thinking of > > > pu

Re: Storage server

2012-09-08 Thread Martin Steigerwald
Am Samstag, 8. September 2012 schrieb Veljko: > On Sat, Sep 08, 2012 at 08:23:36PM +0200, Martin Steigerwald wrote: > > Are you serious about that? > > > > You are planning to mix backup, productions workloads and testing on > > a single *desktop class* machine? > > > > If you had a redundant and

Re: Storage server

2012-09-08 Thread Veljko
On Sat, Sep 08, 2012 at 08:23:36PM +0200, Martin Steigerwald wrote: > Are you serious about that? > > You are planning to mix backup, productions workloads and testing on a > single *desktop class* machine? > > If you had a redundant and failsafe virtualization cluster with 2-3 hosts > and redu

Re: Storage server

2012-09-08 Thread Martin Steigerwald
Am Samstag, 8. September 2012 schrieb Veljko: > On Fri, Sep 07, 2012 at 01:26:13PM -0500, Stan Hoeppner wrote: > > On 9/7/2012 11:29 AM, Veljko wrote: > > > > > > > I'm in the process of making new backup server, so I'm thinking of > > > best way of doing it. I have 4 3TB disks and I'm thinking of

Re: Storage server

2012-09-08 Thread Martin Steigerwald
Am Freitag, 7. September 2012 schrieb Bob Proulx: > Unfortunately I have some recent FUD concerning xfs. I have had some > recent small idle xfs filesystems trigger kernel watchdog timer > recoveries recently. Emphasis on idle. Active filesystems are always > fine. I used /tmp as a large xfs fi

Re: Storage server

2012-09-08 Thread Martin Steigerwald
Am Freitag, 7. September 2012 schrieb Stan Hoeppner: > On 9/7/2012 12:42 PM, Dan Ritter wrote: […] > > Now, the next thing: I know it's tempting to make a single > > filesystem over all these disks. Don't. The fsck times will be > > horrendous. Make filesystems which are the size you need, plus a >

Re: Storage server

2012-09-08 Thread Martin Steigerwald
Am Freitag, 7. September 2012 schrieb Veljko: > > This is Debian. Since 1997 or so, you have had the ability to > > upgrade from major version n to version n+1 without > > reinstalling. You won't need to reinstall unless you change > > architectures (i.e. from x86_32 to x86_64). > But, isn't compl

Re: Storage server

2012-09-08 Thread Veljko
On Fri, Sep 07, 2012 at 01:43:47PM -0600, Bob Proulx wrote: > Veljko wrote: > > Dan Ritter wrote: > > > > OS I would use is Wheezy. Guess he will be stable soon enough and I > > > > don't want to reinstall everything again in one year, when support for > > > > old stable is dropped. > > > > > > Th

Re: Storage server

2012-09-08 Thread Veljko
On Fri, Sep 07, 2012 at 09:43:57PM -0400, tdowg1 news wrote: > >> I'm in the process of making new backup server, so I'm thinking of best > >> way of doing it. I have 4 3TB disks and I'm thinking of puting them in > >> software RAID10. > >> > >> I created 2 500MB partitions for /boot (RAID1) and th

Re: Storage server

2012-09-08 Thread Veljko
On Fri, Sep 07, 2012 at 01:26:13PM -0500, Stan Hoeppner wrote: > On 9/7/2012 11:29 AM, Veljko wrote: > > > I'm in the process of making new backup server, so I'm thinking of best > > way of doing it. I have 4 3TB disks and I'm thinking of puting them in > > software RAID10. > > ["what if" stream

Re: Storage server

2012-09-07 Thread tdowg1 news
>> I'm in the process of making new backup server, so I'm thinking of best >> way of doing it. I have 4 3TB disks and I'm thinking of puting them in >> software RAID10. >> >> I created 2 500MB partitions for /boot (RAID1) and the rest it RAID10. > > So far, so good. > >> LVM will provide me a way t

Re: Storage server

2012-09-07 Thread Bob Proulx
Stan Hoeppner wrote: > Dan Ritter wrote: > > You can put cheap SATA disks in, instead of expensive SAS disks. > > The performance may not be as good, but I suspect you are > > looking at sheer capacity rather than IOPS. > > Stick with enterprise quality SATA disks. Throwing "drive of the week" >

  1   2   >