I dont want to ship big files through puppet. I only want to manage them. We
seed the app data and then it just naturally grows to 500GB+
What puppet ships through (souce =>) is a dataseed which is at most a few
MB.
If you still want to do it:
*) Is puppet still doing this read on the client? yes
*) Is puppet still doing this read on the server? I have not checked, but
the files on the server are tiny.
*) Are you pretty sure it's a full read and not just puppet running stat on
each file to make sure the file(s) are there? Yes, positive its a full read,
the lstats/fstats return quickly. Puppetd starts calling read() on some of
the files under the aforementioned resource.
open("/var/lib/app_data/app_instance/app_id/app_file.foo", O_RDONLY) = 6
fstat(6, {st_mode=S_IFREG|0750, st_size=1048576, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x2b13991b6000
read(6,
"O`:\370\0\0\0\0\377\377\377\377\377\377\377\377\0\0\0/\320l{uE\277\0\0\0\0\0\0"...,
4096) = 4096
read(6,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096)
= 4096
read(6,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096)
= 4096
read(6,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096)
= 4096
On Thu, Jul 15, 2010 at 11:37 AM, Patrick Mohr <kc7...@gmail.com> wrote:
>
> On Jul 15, 2010, at 10:27 AM, John Cesario wrote:
>
> > Hey all,
> >
> > This is on puppet 0.25.4:
> >
> > The manifest for one of the directories looks like this. This
> class/object does not have anything below it (nothing inherits this).
> >
> > file {
> > "/var/lib/data/$name":
> > ensure => directory,
> > owner => "data",
> > group => "data",
> > recurse => true,
> > mode => 750,
> > ignore => ".svn",
> > replace => false,
> > checksum => undef,
> > backup => false,
> > source => "puppet:///app/app_data/$seed",
> > require => File["/var/lib/app_data"]
> > }
> >
> >
> > We use this to initialize an applications data directory, and then dump
> ~500GB of data into it.
> >
> > The problem is that on subsequent puppet runs when the directory is
> populated, strace still shows puppet doing a full read of all the files in
> there.
> >
> > Any way to stop this besides doing recurselimit => 0. I would like the
> permissions to be managed, and obviously with recurse => 0 shipping the seed
> files over there becomes difficult.
>
> Frankly, I would use something other than puppet for big files. I'd
> suggest rsync, nfs, deb packages, or wget+tar.
>
> If you still want to do it:
> *) Is puppet still doing this read on the client?
> *) Is puppet still doing this read on the server?
> *) Are you pretty sure it's a full read and not just puppet running stat on
> each file to make sure the file(s) are there?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com<puppet-users%2bunsubscr...@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
