Hi,
Not sure if I can help but just have couple of questions.
How long does it take to do a 'find' on the directory or a 'stat'?
Have you tried running the client with --verbose --debug --evaltrace
--summarize?
Are you trying to set any permissions inside that directory elsewhere in the
manifest?
I would expect recurse => false not to do anything too, but (and this should
throw an error) can you set recurselimit? If so, does it make any difference?
HTH
Den
On 14/09/2012, at 1:55, Bernd Adamowicz <bernd.adamow...@esailors.de> wrote:
> Hope, the list doesn't blame me for spamming, but I'd just like to provide
> some more information on my problem, still hoping someone can help.
>
> Using the 'iostat' command while Puppet is stuck, I can clearly see that
> there's a very high I/O on the disk:
>
> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz
> avgqu-sz await svctm %util
> sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 0.00 0.00 0.00
> dm-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 0.00 0.00 0.00
> dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 0.00 0.00 0.00
> fioa 0.00 0.00 163.00 5.00 31280.00 40.00 186.43
> 0.04 0.25 0.25 4.20
> dm-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 0.00 0.00 0.00
> dm-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 0.00 0.00 0.00
> dm-4 0.00 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 0.00 0.00 0.00
> dm-5 0.00 0.00 163.00 5.00 31280.00 40.00 186.43
> 0.04 0.25 0.25 4.20
> dm-6 0.00 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 0.00 0.00 0.00
> drbd0 0.00 0.00 163.00 5.00 31280.00 40.00 186.43
> 0.96 0.53 5.72 96.10
> drbd1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
> 0.00 0.00 0.00 0.00
>
> * The 'drbd0' device is an HP IO Accelerator card
> * The 'rsec/s' column for drbd0 shows the number of sectors read from disk
> per second. This value is pretty low if Puppet is not running.
> * The '%util' (CPU time during the I/O requests) column for drbd0 shows that
> the device is almost in saturation
> * This is the mount point for drbd0: '/dev/ drbd0 on /repository' and
> '/repository/sonatype-work' is the directory hosted by Puppet
>
> Again, this is the configuration:
>
> file {
> "/repository/sonatype-work":
> ensure => directory,
> owner => $nexus_user,
> group => $nexus_group,
> mode => 0755,
> recurse => false,
> backup => false,
> }
>
> For me it still seems there's a kind of recursive scanning done by Puppet
> though 'recurse' is set to false.
>
> Thanks for helping,
> Bernd
>
> Von: puppet-users@googlegroups.com [mailto:puppet-users@googlegroups.com] Im
> Auftrag von Bernd Adamowicz
> Gesendet: Donnerstag, 13. September 2012 14:45
> An: 'puppet-users@googlegroups.com'
> Betreff: AW: [Puppet Users] AW: Issue with large directory content
>
> This keeps being weird. Simply thought to wait until Puppet finishes, but had
> to quit after one and a half hour. Also tried 'ensure => present' instead of
> 'ensure => directory' with no success. No log output at all. Still
> investigating. But any ideas still highly appreciated!
>
> Bernd
>
> Von: puppet-users@googlegroups.com [mailto:puppet-users@googlegroups.com] Im
> Auftrag von Bernd Adamowicz
> Gesendet: Donnerstag, 13. September 2012 10:46
> An: puppet-users@googlegroups.com
> Betreff: Re: [Puppet Users] AW: Issue with large directory content
>
> Thanks for your answers so far.
>
> But beware that the huge artifacts are *not* managed by Puppet (see recurse
> => false). Actually it's a Maven repository filled by Nexus. Only the top
> directory is managed by Puppet to have it in place and have correct access
> rights. This worked well until I initially filled the repository with the
> artifacts manually. That slowed down Puppet. Seems to me as Puppet will do
> some recursive scanning, but that's just an assumption, since Puppet is
> running with almost 100% CPU load.
>
> I turned debug on in Puppet but will not see anything even after a few
> minutes. Presumably I would see something if I let Puppet run just long
> enough. However, it's a strange behaviour I've never experienced. And I think
> my configuration is OK.
>
> Bernd
>
> On 09/13/2012 01:24 AM, Peter Brown wrote:
> On 13 September 2012 00:12, Christopher Wood <christopher_w...@pobox.com>
> wrote:
>> I don't have enough information to say. You might want to run the master and
>> agent in debug mode to get more output, though.
>>
>> puppet agent --debug --verbose --no-daemonize
>>
>> Also, 100 GB? Any particular reason why you're not installing this using a
>> content distribution system or a large number of RPMs?
>
> Recursing through A 100Gb directory will definitely slow down your puppet run.
> If the contents of the directory are reasonably static an RPM would be
> the best idea.
> If it's not static a git or svn repo would be a better idea.
> If you are tricky you can manage the checkouts of git or svn with
> puppet as well.
> I wrote a few tricky resources for this a while ago and they are
> infinitely handy.
>
>>
>> On Wed, Sep 12, 2012 at 04:08:13PM +0200, Bernd Adamowicz wrote:
>>> No ideas at all?
>>>
>>>> -----Ursprüngliche Nachricht-----
>>>> Von: Bernd Adamowicz
>>>> Gesendet: Dienstag, 11. September 2012 16:16
>>>> An: puppet-users@googlegroups.com
>>>> Betreff: Issue with large directory content
>>>>
>>>> Hi all,
>>>>
>>>> I got this directory configuration:
>>>>
>>>>
>>>> file {
>>>> "${codebase_ng::repository_mount}/${sonatype_work_dir}":
>>>> ensure => directory,
>>>> owner => $nexus_user,
>>>> group => $nexus_group,
>>>> mode => 0755,
>>>> recurse => false,
>>>> backup => false,
>>>> }
>>>>
>>>> Today I added some 100GB of artifacts to a subdirectory of
>>>> "${codebase_ng::repository_mount}/${sonatype_work_dir}". Now the result
>>>> is that the Puppet seems to run "forever". If I uncomment this code,
>>>> Puppet finishes in 15 seconds. So I presume Puppet is doing some
>>>> recursive scanning of this directory. Could this be true? Is there a
>>>> know issue with large content of directories?
>>>>
>>>> Thanks in advance!
>>>> Bernd
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
