On Tue, 12 Feb 2013 12:10:15 -0600
Serge Hallyn <serge.hal...@ubuntu.com> wrote:
> Quoting Dwight Engen (dwight.en...@oracle.com):
> > On Mon, 11 Feb 2013 16:40:39 -0500
> > Stéphane Graber <stgra...@ubuntu.com> wrote:
> >
> > > On 02/11/2013 03:43 PM, Serge Hallyn wrote:
> > > >
> > > > The previous lxcpath patches added support for a custom LXCPATH
> > > > set through a system-wide configuration file.
> > > >
> > > > This was also exposed through the C api, so that a custom
> > > > lxcpath could be set at the container object instanciation
> > > > time, or set at runtime.
> > > >
> > > > However the command sock filename was always located under the
> > > > global lxcpath, which could be confusing, and would be a problem
> > > > for users with insufficient perms to the system-wide lxc path
> > > > (i.e. if setting lxcpath to $HOME/lxcbase). This patch changes
> > > > that by passing the lxcpath to all callers of lxc_command().
> > > >
> > > > It remains to add an lxcpath command line argument to most of
> > > > the command line tools (which are not using the C api) -
> > > > lxc-start, lxc-info, lxc-stop, etc.
> > > >
> > > > At this point it becomes tempting to do something like
> > > >
> > > > c = lxc.Container("r1", "/var/lib/lxc")
> > > > c2 = lxc.Container("r1", "$HOME/lxcbase")
> > > >
> > > > However, that's problematic - those two will use the same
> > > > directory names for cgroup directories.
> > > >
> > > > What would be the best way to handle this? One way (which I
> > > > kind of like) is to give up on naming the cgroups after the
> > > > container. use mkstemp for the cgroup name, let lxc keep track
> > > > of the cgroup name based on the command socket, and make users
> > > > use lxc-cgroup to get and change settings.
> > >
> > > As we discussed on IRC, I think it'd be best to do two things:
> > > 1) Add a new lxc.cgroup.directory option which would default to
> > > "lxc" (our current directory) and let people easily override to
> > > another name to avoid the conflict.
> >
> > I like this, and I think it might also allow for groups of
> > containers I was refering to in irc with a little more support in
> > the config file for setting limits at the directory level (at least
> > to a grouping depth of 1). For example lets say a config file looks
> > like:
> >
> > lxc.cgroup.directory = builds
> > lxc.cgroup.directory.memory.limit_in_bytes = 1073741824
> >
> > This would set /sys/fs/cgroup/memory/builds/memory.limit_in_bytes
> > which would limit all containers in the "group" builds together to
> > 1G. Any
>
> I was thinking, rather, that
>
> lxc.cgroup.directory = builds
>
> with a container name of r1 would result in
>
> /sys/fs/cgroup/memory/builds/r1/memory.limit_in_bytes.
>
> And, if I simultaneously run another r1 from a different lxcpath,
>
> /sys/fs/cgroup/memory/builds/r1-1/memory.limit_in_bytes.
I agree! I'm assuming the second r1 (r1-1) also had
lxc.cgroup.directory = builds. I think I wasn't clear, but what I was
wondering was how to allow setting the upper level:
/sys/fs/cgroup/memory/builds/memory.limit_in_bytes
in order to limit 'r1' + 'r1-1' memory usage. It would be nice to be
able to set this from lxc config, but I guess it can be done
afterwards so we don't need it right away. The use case is something
like "lxc-start N number of build containers each with a different
version of $distro and only let them use XX amount of memory in total
since this host machine is also used for [something]...". So r1 and r1-1
would not have a memory limit, but builds would, limiting the aggregate
of all containers under builds.
> This does conflict a bit with the idea of libcgroup-pam putting the
> user into his own cgroups, which then confine the containers he
> creates.
>
> For the general case, I far prefer the libcgroup-pam-based solution,
> because we can set up intelligent defaults (from a distro level),
> easily overrideable by an interested admin. And it lets pam (with
> privilege) create and set up the cgroups, chown them to the user so
> he can create new cgroups under that.
I think pam based for users makes sense, but I like the "system" ones
pretty much the way they are now :)
> I'm not 100% sure yet how to mix this with the lxc.cgroup.directory.
> Unfortunately hierarchy in cgroups has a (reputedly - I haven't tested
> myself) steep performance cost, so we don't just want to default to
> /sys/fs/cgroup/memory/hallyn/builds/r1/...
>
> I think it makes sense to say that lxc.cgroup.directory requires
> privilege, and always groups at the top level. Each distro can have
> an init/upstart/systemd script which sets up configured groups, so
> that then on my system I might have under /sys/fs/cgroup/memory/:
>
> builds/r1
> builds/r1-2
> builds/r1-3
> hallyn/r1
> hallyn/r1-1
I agree about lxc.cgroup.directory needing privilege and being at the
top level (may help alleviate performance hit too). So just to make
sure, I think we're saying it would be:
With privilege, lxc.cgroup.directory not specified (just like things
are now, but with the possibility of r1-1 when using multiple config
paths):
/sys/fs/cgroup/memory/lxc/r1/memory.limit_in_bytes
/sys/fs/cgroup/memory/lxc/r1-1/memory.limit_in_bytes
With privilege, lxc.cgroup.directory = builds:
/sys/fs/cgroup/memory/builds/r1/memory.limit_in_bytes
/sys/fs/cgroup/memory/builds/r1-1/memory.limit_in_bytes
Without privilege, user hallyn:
/sys/fs/cgroup/memory/hallyn/r1/memory.limit_in_bytes
/sys/fs/cgroup/memory/hallyn/r1-1/memory.limit_in_bytes
> This also gives us clear places to configure limits for all the
> groups - libcgroup-pam for hallyn, and the lxc init script for builds.
Sounds good to me.
> thoughts?
>
> -serge
------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
_______________________________________________
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel
