RE: Question about rsync and BIG mirror

2006-03-06 Thread johan.boye
> Object: Re: Question about rsync and BIG mirror Thanks for all your answers and advices. My problem seems on the side of the 2MB line one time the whole 190GB data are synchronised. I will keep in touch and give some feedbacks. Thanks for all -- To unsubscribe or change options: ht

Re: Question about rsync and BIG mirror

2006-03-06 Thread Jamie Lokier
Shachar Shemesh wrote: > >While you're there, one little trick I've found that speeds up > >scanning large directory hierarchies is to stat() or open() entries in > >inode-number order. For some filesystems it makes no difference, but > >for others it reduces the average disk seek time as on many

Re: Question about rsync and BIG mirror

2006-03-06 Thread Shachar Shemesh
Jamie Lokier wrote: >While you're there, one little trick I've found that speeds up >scanning large directory hierarchies is to stat() or open() entries in >inode-number order. For some filesystems it makes no difference, but >for others it reduces the average disk seek time as on many common >fi

Re: Question about rsync and BIG mirror

2006-03-06 Thread Jamie Lokier
Wayne Davison wrote: > On Mon, Mar 06, 2006 at 07:18:45PM +0200, Shachar Shemesh wrote: > > In fact, I know of at least one place where they don't use rsync because > > they don't have enough RAM+SWAP to hold the list of files in memory. > > > > As far as future directions for rsync, I think this

Re: Question about rsync and BIG mirror

2006-03-06 Thread Jamie Lokier
Shachar Shemesh wrote: > >Hmm. My home directory, on my laptop (a mere 60GB disk), does contain > >millions of files, and it takes about 20 minutes to build the list on > >a good day. 100Mbps network, but it's I/O bound not network bound. > > > >It looks a lot like the number of files is more sig

Re: Question about rsync and BIG mirror

2006-03-06 Thread Wayne Davison
On Mon, Mar 06, 2006 at 07:18:45PM +0200, Shachar Shemesh wrote: > In fact, I know of at least one place where they don't use rsync because > they don't have enough RAM+SWAP to hold the list of files in memory. > > As far as future directions for rsync, I think this is the major place > where rsyn

Re: Question about rsync and BIG mirror

2006-03-06 Thread Shachar Shemesh
Jamie Lokier wrote: >Hmm. My home directory, on my laptop (a mere 60GB disk), does contain >millions of files, and it takes about 20 minutes to build the list on >a good day. 100Mbps network, but it's I/O bound not network bound. > >It looks a lot like the number of files is more significant tha

Re: Question about rsync and BIG mirror

2006-03-06 Thread Jamie Lokier
jp wrote: > 100gb of 4-40MB files sounds like my home PC full of digital photos I've > taken. It backs up to a linux PC right beside it with rsync. I don't > really call it that big a project for rsync. Big things for rsync are > millions of files. At 100mbps, it takes a few seconds to build the

Re: Question about rsync and BIG mirror

2006-03-06 Thread jp
100gb of 4-40MB files sounds like my home PC full of digital photos I've taken. It backs up to a linux PC right beside it with rsync. I don't really call it that big a project for rsync. Big things for rsync are millions of files. At 100mbps, it takes a few seconds to build the list. I use the

TR: Question about rsync and BIG mirror

2006-03-06 Thread johan.boye
> -Message d'origine- > De : BOYE Johan > Envoyé : lundi 6 mars 2006 08:28 > À : 'Jan-Benedict Glaw' > Objet : RE: Question about rsync and BIG mirror > > > > I'm preparing a plan for a production mode in my company: we need to >

RE: Question about rsync and BIG mirror

2006-03-03 Thread Tony
; From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of > [EMAIL PROTECTED] > Sent: Friday, March 03, 2006 1:03 AM > To: rsync@lists.samba.org > Subject: Question about rsync and BIG mirror > > > // I wonder if this message has been posted, so I sent it again // > &

Re: Question about rsync and BIG mirror

2006-03-03 Thread Jan-Benedict Glaw
On Fri, 2006-03-03 08:02:55 +0100, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > // I wonder if this message has been posted, so I sent it again // It was, but nobody answered yet. > I'm preparing a plan for a production mode in my company: we need to > mirror around 100GB of data trough a spe

Re: Question about rsync and BIG mirror

2006-03-03 Thread Shachar Shemesh
[EMAIL PROTECTED] wrote: >Hello, > > So: each night, from 0:00am to maximum 7:00am, the server will have to >check the 100Go of files and see what files have been modified, then, >upload them to the clients. Each file is around 4MB to 40MB in average. > > Are the clients what you call the "mir

Question about rsync and BIG mirror

2006-03-02 Thread johan.boye
// I wonder if this message has been posted, so I sent it again // Hello, I'm quite a n00b on rsync stuff but I went to the website, read FAQ/how-to, Google and more, I setup my own rsync server and clients: everything works fine :-D I'm preparing a plan for a production mode in my company:

Question about rsync and BIG mirror

2006-03-02 Thread johan.boye
Hello, I'm quite a n00b on rsync stuff but I went to the website, read FAQ/how-to, Google and more, I setup my own rsync server and clients: everything works fine :-D I'm preparing a plan for a production mode in my company: we need to mirror around 100GB of data trough a special VPN internet