On Fri, 2010-03-12 at 03:30 -0800, DieterVDW wrote: > On Mar 12, 11:21 am, Patrick <kc7...@gmail.com> wrote: > > Puppet doesn't handle a folder with lots of files well. It handles > large files even worse. The standard advice is "Try putting the files > in a package and distributing them using apt." Another common answer > is to try combining exec and rsync. I ended up using apt. Here are > the tutorials I used: > > The problem is, I -am- using apt! > Those files are downloaded and installed using apt, I just want puppet > to make sure they are owned by a certain user and group. > That's the only thing puppet needs to do. > > If I do the same in bash: > find directory/ | while read file ; do chown user:group "$file" ; done > > real 0m28.119s > user 0m4.064s > sys 0m12.725s > > I can live with overhead from slow ruby, etc etc ... > But apparently Puppet is 60x slower than bash for doing the same > action! > > During my investigation for this problem I've seen a lot of people > saying things like: > "Puppet doesn't handle a folder with lots of files well" > 60x times slower isn't really "not handling well". It's crap. > I'm a bit baffled by the defeatism I see in the Puppet community > concerning Puppet CPU usage. > > I can't really believe people would call what I am experiencing "not > handling well", so I suppose I'm having another, worse, issue > concerning my setup? > > Also the usual suspect for puppet not handling large file collections > well seems to be the checksumming. > But with that turned of, what's keeping puppet busy?
This is a known (at least to me) regression in 0.25: Puppet still uses md5 checksums on every managed files, even when it is not needed. > Puppet eats up 30 minutes of CPU time, I see two options: > - Puppet is doing something necessary for it's functioning during that > time, in which case somebody should know what it is doing. Anybody? It is checksumming every file. > - Or the code is gravely flawed and needs to be fixed. This is a bug, but I won't say it is gravely flawed, because in the end you have the correct result. > Is this a stroke of bad luck, or should I conclude that Puppet isn't > production ready? Puppet is production ready, as many here are running it on critical production systems or on big infrastructure. It's just that like every piece of software it can contain small defects that needs to be reported and fixed. > I really can't have this on production machines... I don't see why. Would you throw the complete software and revert managing everything by hand because you encountered a small performance issue for which you can find correct work-arounds, like an exec of a recursive chmod? -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
