Hi, Is this directory the only resource you are managing or are there others. If not then try disabling all other resources and see if you still get the same issue?
Best Regards Martin Wheldon On Fri, Mar 12, 2010 at 12:55 PM, DieterVDW <dieter...@gmail.com> wrote: > Also, whatever Puppetd is doing then, it seems to be unnecessary. > > If I execute puppetd -tv and wait until all files have been chown'd, > and then hit CTRL-C and run puppetd again, > the second time, puppetd finishes in about a minute. > The total process of running puppetd twice takes about 4 minutes . > > So by manipulating the execution of puppetd using a CTRL-C keystroke I > can reach the intended end state in 4 minutes. > If I leave puppetd to do it by itself, it takes 30+ minutes of 100% > CPU usage... > > On Mar 12, 1:43 pm, DieterVDW <dieter...@gmail.com> wrote: >> Aren't the time values in seconds? >> Because I can't match the "Total: 54.78" line to the "notice: Finished >> catalog run in 1877.06 seconds" . >> >> Also, the strange thing is that the actual changing of owner and group >> happens pretty fast. >> I see these messages passing during +- 20s for all files: >> notice: /File[/some/file]/owner: owner changed 'wronguser' to >> 'rightuser' >> notice: /File[/some/file]/group: group changed 'wronggroup' to >> 'rightgroup' >> >> It's AFTER this has happened that puppetd stalls for 30+ minutes. >> So it must be some kind of wrapup action... ? >> >> D >> >> On Mar 12, 1:26 pm, Trevor Vaughan <tvaug...@onyxpoint.com> wrote: >> >> > So, it looks like 30 minutes is being taken up in the Package type and >> > 19 in the File type. >> >> > The Package type probably won't eat up much CPU at all as it's just >> > fetching and installing packages (unless they are some huge packages). >> >> > You seem to be trying to recursively manage *something* using the File >> > type. >> >> > What happens is that Puppet creates an actual file object for each >> > file and directory in the recursed path. Then, it takes the graph and >> > applies the changes to each file that has been seen. I'm not certain, >> > but I think that it builds this tree each time it runs to ensure that >> > nothing has changed under the path. >> >> > This processing has to happen on the client since the server has no >> > previous knowledge of the client's file structure. >> >> > That said, for deep directory/many file file statements, just spawn an >> > exec until the internal Puppet logic can be optimized. >> >> > Trevor >> >> > On Fri, Mar 12, 2010 at 6:32 AM, DieterVDW <dieter...@gmail.com> wrote: >> > > Some more information from the (undocumented) --summarize option I >> > > just discovered: >> >> > > Changes: >> > > Total: 4271 >> > > Resources: >> > > Applied: 4271 >> > > Out of sync: 2138 >> > > Scheduled: 4435 >> > > Total: 115 >> > > Time: >> > > Config retrieval: 1.36 >> > > Exec: 0.77 >> > > File: 19.23 >> > > Filebucket: 0.00 >> > > Host: 0.00 >> > > Package: 31.99 >> > > Schedule: 0.00 >> > > Service: 1.42 >> > > User: 0.01 >> > > Total: 54.78 >> > > warning: Value of 'preferred_serialization_format' (pson) is invalid >> > > for report, using default (marshal) >> > > notice: Finished catalog run in 1877.06 seconds >> >> > > It seems to me the cause of the delays is not recorded in the time >> > > overview? >> >> > > D >> >> > > On Mar 12, 12:30 pm, DieterVDW <dieter...@gmail.com> wrote: >> > >> On Mar 12, 11:21 am, Patrick <kc7...@gmail.com> wrote: >> >> > >> > Puppet doesn't handle a folder with lots of files well. It handles >> > >> > large files even worse. The standard advice is "Try putting the >> > >> > files in a package and distributing them using apt." Another common >> > >> > answer is to try combining exec and rsync. I ended up using apt. >> > >> > Here are the tutorials I used: >> >> > >> The problem is, I -am- using apt! >> > >> Those files are downloaded and installed using apt, I just want puppet >> > >> to make sure they are owned by a certain user and group. >> > >> That's the only thing puppet needs to do. >> >> > >> If I do the same in bash: >> > >> find directory/ | while read file ; do chown user:group "$file" ; done >> >> > >> real 0m28.119s >> > >> user 0m4.064s >> > >> sys 0m12.725s >> >> > >> I can live with overhead from slow ruby, etc etc ... >> > >> But apparently Puppet is 60x slower than bash for doing the same >> > >> action! >> >> > >> During my investigation for this problem I've seen a lot of people >> > >> saying things like: >> > >> "Puppet doesn't handle a folder with lots of files well" >> > >> 60x times slower isn't really "not handling well". It's crap. >> > >> I'm a bit baffled by the defeatism I see in the Puppet community >> > >> concerning Puppet CPU usage. >> >> > >> I can't really believe people would call what I am experiencing "not >> > >> handling well", so I suppose I'm having another, worse, issue >> > >> concerning my setup? >> >> > >> Also the usual suspect for puppet not handling large file collections >> > >> well seems to be the checksumming. >> > >> But with that turned of, what's keeping puppet busy? >> >> > >> Puppet eats up 30 minutes of CPU time, I see two options: >> > >> - Puppet is doing something necessary for it's functioning during that >> > >> time, in which case somebody should know what it is doing. Anybody? >> > >> - Or the code is gravely flawed and needs to be fixed. >> >> > >> Is this a stroke of bad luck, or should I conclude that Puppet isn't >> > >> production ready? >> > >> I really can't have this on production machines... >> >> > > -- >> > > You received this message because you are subscribed to the Google >> > > Groups "Puppet Users" group. >> > > To post to this group, send email to puppet-us...@googlegroups.com. >> > > To unsubscribe from this group, send email to >> > > puppet-users+unsubscr...@googlegroups.com. >> > > For more options, visit this group >> > > athttp://groups.google.com/group/puppet-users?hl=en. >> >> > -- >> > Trevor Vaughan >> > Vice President, Onyx Point, Inc >> > (410) 541-6699 >> > tvaug...@onyxpoint.com >> >> > -- This account not approved for unencrypted proprietary information -- > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-us...@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
