Re: [lxc-devel] [PATCH] Add mechanism for container to notify host about end of boot
Hi there, I've now updated my patch, there are now the following changes, partially based on feedback from this list, partially from sorting things in my head a bit. - socket instead of a FIFO - is now in /run instead of /dev - parent directories of socket inside container are automatically created - extensible but very simple protocol, line-based, currently, only 'status NEWSTATUS' is supported, where NEWSTATUS may be either RUNNING or STOPPING - now returns either 'OK' or 'ERR message' to the caller Regards, Christian -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
[lxc-devel] [PATCH v2] Add mechanism for container to notify host about end of boot
This patch adds a simple notification system that allows the container to
notify the host (in particular, the lxc-start process) that the boot process
has been completed successfully. It also adds an additional status BOOTING
that lxc-info may return. This allows the administrator and scripts to
distinguish between a fully-running container and a container that is still
in the process of booting.
If nothing is added to the configuration file, the current behavior is not
changed, i.e. after lxc-start finishes the initialization, the container is
immediately put into the RUNNING state. This ensures backwards
compatibility.
If lxc.notification.enabled is set to yes, after lxc-start initialization
the container is initially put into the state BOOTING. Also, a UNIX domain
socket /var/lib/lxc/%s/notification-socket is created and bind-mounted into
the container, by default to /run/lxc-notify. (This can be changed by the
lxc.notification.path setting.) The default access mode of this socket is
0600, so only root may access it.
A program in the container may connect to the socket and use a very simple
protocol to notify lxc-start that the boot sequence of the container has
been completed. Sending 'status RUNNING' or 'status STOPPING' will cause the
container to change the status to either RUNNING or STOPPING, others are not
allowed from within the container. If everything succeeds, lxc-start will
respond with 'OK', otherwise with 'ERR $error_message'.
The following script may be used from the shell to notify lxc-start from
within the container that boot has completed successfully:
echo status RUNNING | perl -e \
'use IO::Socket;
$client = IO::Socket::UNIX->new(
Peer => "/run/lxc-notify",
Type => SOCK_STREAM,
Timeout => 10
);
while (<>) {
print $client $_;
}'
If OpenBSD's version of netcat is available, executing
echo status RUNNING | nc -U /run/lxc-notify
is also possible. In addition to RUNNING, STOPPING is also allowed from
within the container, so if somebody types 'shutdown -h now' in the
container, the status gets updated immediately.
The mechanism is designed to be extensible, i.e. commands other than
'status' may be supported at a later point, if other needs arise for the
communication between inside and outside the container.
Signed-off-by: Christian Seiler
Cc: Serge Hallyn
---
src/lxc/Makefile.am|1 +
src/lxc/conf.c |8 +
src/lxc/conf.h |2 +
src/lxc/confile.c | 35
src/lxc/notification.c | 435
src/lxc/notification.h | 40 +
src/lxc/start.c| 22 ++-
src/lxc/start.h|1 +
src/lxc/state.c|1 +
src/lxc/state.h|3 +-
10 files changed, 544 insertions(+), 4 deletions(-)
create mode 100644 src/lxc/notification.c
create mode 100644 src/lxc/notification.h
diff --git a/src/lxc/Makefile.am b/src/lxc/Makefile.am
index 7d86ad6..d976bf7 100644
--- a/src/lxc/Makefile.am
+++ b/src/lxc/Makefile.am
@@ -32,6 +32,7 @@ liblxc_so_SOURCES = \
freezer.c \
checkpoint.c \
restart.c \
+ notification.h notification.c \
error.h error.c \
parse.c parse.h \
cgroup.c cgroup.h \
diff --git a/src/lxc/conf.c b/src/lxc/conf.c
index f3c2334..7c9e936 100644
--- a/src/lxc/conf.c
+++ b/src/lxc/conf.c
@@ -61,6 +61,7 @@
#include "log.h"
#include "lxc.h" /* for lxc_cgroup_set() */
#include "caps.h" /* for lxc_caps_last_cap() */
+#include "notification.h"
#if HAVE_APPARMOR
#include
@@ -2278,6 +2279,11 @@ int lxc_setup(const char *name, struct lxc_conf
*lxc_conf)
return -1;
}
+ if (lxc_notification_mount_hook(name, lxc_conf)) {
+ ERROR("failed to init notification mechanism for container
'%s'.", name);
+ return -1;
+ }
+
if (setup_cgroup(name, &lxc_conf->cgroup)) {
ERROR("failed to setup the cgroups for '%s'", name);
return -1;
@@ -2583,6 +2589,8 @@ void lxc_conf_free(struct lxc_conf *conf)
if (conf->aa_profile)
free(conf->aa_profile);
#endif
+ if (conf->notification_path)
+ free(conf->notification_path);
lxc_clear_config_caps(conf);
lxc_clear_cgroups(conf, "lxc.cgroup");
lxc_clear_hooks(conf, "lxc.hook");
diff --git a/src/lxc/conf.h b/src/lxc/conf.h
index dccc176..af9a178 100644
--- a/src/lxc/conf.h
+++ b/src/lxc/conf.h
@@ -237,6 +237,8 @@ struct lxc_conf {
#endif
char *seccomp; // filename with the seccomp rules
int maincmd_fd;
+ int notification_enabled;
+ char *notification_path;
};
int run_lxc_hooks(const char *name, char *hook, struct lxc_conf *conf);
diff --git a/src/lxc/confile.c b/src/lxc/confile.c
index 94bb698..fa15fd5 100644
--- a/src/lxc/confile.c
+++ b/src/lxc/confile.c
@@ -53,6 +53,8 @@ static int config_ttydir(const char *, char *, struct
lxc_conf *
Re: [lxc-devel] [PATCH] Add mechanism for container to notify host about end of boot
On 12-09-19 07:49 AM, Christian Seiler wrote: > Hi there, > > I've now updated my patch, there are now the following changes, > partially based on feedback from this list, partially from sorting > things in my head a bit. > > - socket instead of a FIFO > - is now in /run instead of /dev I don't think that part's going to work... Most distros mount /run as tmpfs at boot time which will hide anything you're putting in there before boot. > - parent directories of socket inside container are automatically > created > - extensible but very simple protocol, line-based, currently, only > 'status NEWSTATUS' is supported, where NEWSTATUS may be either > RUNNING or STOPPING > - now returns either 'OK' or 'ERR message' to the caller > > Regards, > Christian -- Stéphane Graber Ubuntu developer http://www.ubuntu.com signature.asc Description: OpenPGP digital signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] Add mechanism for container to notify host about end of boot
Hi, >> - is now in /run instead of /dev > > I don't think that part's going to work... Most distros mount /run as > tmpfs at boot time which will hide anything you're putting in there > before boot. Hmmm, that is indeed a problem... Do you have any suggestions? Or should we just keep it in /dev for now (path is configurable anyway) and worry about this later? Christian -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] Add mechanism for container to notify host about end of boot
On 12-09-19 10:12 AM, Christian Seiler wrote: > Hi, > >>> - is now in /run instead of /dev >> >> I don't think that part's going to work... Most distros mount /run as >> tmpfs at boot time which will hide anything you're putting in there >> before boot. > > Hmmm, that is indeed a problem... Do you have any suggestions? Or > should we just keep it in /dev for now (path is configurable anyway) > and worry about this later? > > Christian I think /dev is the safest at the moment. Arguably it's wrong as it's not an actual device node, but it's the only directory that lxc already requires all distros not to mess with (or we wouldn't have working console, tty, ...). -- Stéphane Graber Ubuntu developer http://www.ubuntu.com signature.asc Description: OpenPGP digital signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH 1/2] fix trivial off by one error
On Tue, 18 Sep 2012 14:32:02 -0700 (PDT)
Christian Seiler wrote:
> Hi,
>
> > Do you think mallocing an fd_set and using FD_SET() and friends
> > would be better? The (dispose|finish) loops would visit FD_SETSIZE
> > bits with an FD_ISSET() test, which is more work than you have
> > currently with the early out, but we would probably save on the
> > initialization with FD_ZERO(). I don't know if
> > lxc_cgroup_(dispose|finish)_attach is performance critical.
>
> I don't think performance is that much of an issue here, but to me it
> seems that using fd_set logic would complicate things quite a bit
> unnecessarily. The current logic is already a bit complicated because
> the cgroup task files have to be opened before setns() but written to
> only after the fork() call when we know the pid which happens after
> setns(). Having a simple array with a loop over it appears to be much
> more straight-forward to me, especially since iterating over an fd_set
> is kind of convoluted.
>
> > Or I can just add a comment :)
>
> My suggestion would be to do just that unless someone has a good
> reason to change the current logic.
>
> All IMHO of course, I just wrote the initial patch, in the end other
> people get to decide what goes in. ;-)
>
> Regards,
> Christian
Just to make clear what I was suggesting, here is a patch. It may not
be any better/clearer than the original code, its less storage (1024
bytes vs 128) and handles more fd's (not that there will ever be that
many), but its more code.
diff --git a/src/lxc/cgroup.c b/src/lxc/cgroup.c
index a02ebc2..63eac93 100644
--- a/src/lxc/cgroup.c
+++ b/src/lxc/cgroup.c
@@ -31,6 +31,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -322,8 +323,8 @@ static int lxc_one_cgroup_attach(const char *name, struct
mntent *mntent, pid_t
int lxc_cgroup_dispose_attach(void *data)
{
- int *fds = data;
- int ret, err;
+ fd_set *fds = data;
+ int fd, ret, err;
if (!fds) {
return 0;
@@ -331,10 +332,12 @@ int lxc_cgroup_dispose_attach(void *data)
ret = 0;
- for (; *fds >= 0; fds++) {
- err = lxc_one_cgroup_dispose_attach(*fds);
- if (err) {
- ret = err;
+ for (fd = 0; fd < FD_SETSIZE; fd++) {
+ if (FD_ISSET(fd, fds)) {
+ err = lxc_one_cgroup_dispose_attach(fd);
+ if (err) {
+ ret = err;
+ }
}
}
@@ -345,21 +348,22 @@ int lxc_cgroup_dispose_attach(void *data)
int lxc_cgroup_finish_attach(void *data, pid_t pid)
{
- int *fds = data;
- int err;
+ fd_set *fds = data;
+ int fd, err;
if (!fds) {
return 0;
}
- for (; *fds >= 0; fds++) {
- err = lxc_one_cgroup_finish_attach(*fds, pid);
- if (err) {
- /* get rid of the rest of them */
- lxc_cgroup_dispose_attach(data);
- return -1;
+ for (fd = 0; fd < FD_SETSIZE; fd++) {
+ if (FD_ISSET(fd, fds)) {
+ err = lxc_one_cgroup_finish_attach(fd, pid);
+ if (err) {
+ /* get rid of the rest of them */
+ lxc_cgroup_dispose_attach(data);
+ return -1;
+ }
}
- *fds = -1;
}
free(data);
@@ -373,9 +377,9 @@ int lxc_cgroup_prepare_attach(const char *name, void **data)
FILE *file = NULL;
int err = -1;
int found = 0;
- int *fds;
+ int fd;
+ fd_set *fds;
int i;
- static const int MAXFDS = 256;
file = setmntent(MTAB, "r");
if (!file) {
@@ -386,19 +390,17 @@ int lxc_cgroup_prepare_attach(const char *name, void
**data)
/* create a large enough buffer for all practical
* use cases
*/
- fds = malloc(sizeof(int) * MAXFDS);
+ fds = malloc(sizeof(*fds));
if (!fds) {
err = -1;
goto out;
}
- for (i = 0; i < MAXFDS; i++) {
- fds[i] = -1;
- }
+ FD_ZERO(fds);
err = 0;
i = 0;
while ((mntent = getmntent(file))) {
- if (i >= MAXFDS - 1) {
+ if (i >= FD_SETSIZE) {
ERROR("too many cgroups to attach to, aborting");
lxc_cgroup_dispose_attach(fds);
errno = ENOMEM;
@@ -416,12 +418,13 @@ int lxc_cgroup_prepare_attach(const char *name, void
**data)
INFO("[%d] found cgroup mounted at '%s',opts='%s'",
++found, mntent->mnt_dir, mntent->mnt_opts);
- fds[i] = lxc_one_cgroup_prepare_attach(name, mntent);
- if (fds[i] < 0) {
-
[lxc-devel] [PATCH v2.1] Add mechanism for container to notify host about end of boot
This patch adds a simple notification system that allows the container to
notify the host (in particular, the lxc-start process) that the boot process
has been completed successfully. It also adds an additional status BOOTING
that lxc-info may return. This allows the administrator and scripts to
distinguish between a fully-running container and a container that is still
in the process of booting.
If nothing is added to the configuration file, the current behavior is not
changed, i.e. after lxc-start finishes the initialization, the container is
immediately put into the RUNNING state. This ensures backwards
compatibility.
If lxc.notification.enabled is set to yes, after lxc-start initialization
the container is initially put into the state BOOTING. Also, a UNIX domain
socket /var/lib/lxc/%s/notification-socket is created and bind-mounted into
the container, by default to /dev/lxc-notify. (This can be changed by the
lxc.notification.path setting.) The default access mode of this socket is
0600, so only root may access it.
A program in the container may connect to the socket and use a very simple
protocol to notify lxc-start that the boot sequence of the container has
been completed. Sending 'status RUNNING' or 'status STOPPING' will cause the
container to change the status to either RUNNING or STOPPING, others are not
allowed from within the container. If everything succeeds, lxc-start will
respond with 'OK', otherwise with 'ERR $error_message'.
The following script may be used from the shell to notify lxc-start from
within the container that boot has completed successfully:
echo status RUNNING | perl -e \
'use IO::Socket;
$client = IO::Socket::UNIX->new(
Peer => "/dev/lxc-notify",
Type => SOCK_STREAM,
Timeout => 10
);
while (<>) {
print $client $_;
}'
If OpenBSD's version of netcat is available, executing
echo status RUNNING | nc -U /dev/lxc-notify
is also possible. In addition to RUNNING, STOPPING is also allowed from
within the container, so if somebody types 'shutdown -h now' in the
container, the status gets updated immediately.
The mechanism is designed to be extensible, i.e. commands other than
'status' may be supported at a later point, if other needs arise for the
communication between inside and outside the container.
Signed-off-by: Christian Seiler
Cc: Serge Hallyn
---
doc/lxc.conf.sgml.in | 67
src/lxc/Makefile.am|1 +
src/lxc/conf.c |8 +
src/lxc/conf.h |2 +
src/lxc/confile.c | 35
src/lxc/notification.c | 435
src/lxc/notification.h | 40 +
src/lxc/start.c| 22 ++-
src/lxc/start.h|1 +
src/lxc/state.c|1 +
src/lxc/state.h|3 +-
11 files changed, 611 insertions(+), 4 deletions(-)
create mode 100644 src/lxc/notification.c
create mode 100644 src/lxc/notification.h
diff --git a/doc/lxc.conf.sgml.in b/doc/lxc.conf.sgml.in
index 1428f25..a4950e8 100644
--- a/doc/lxc.conf.sgml.in
+++ b/doc/lxc.conf.sgml.in
@@ -738,6 +738,73 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
02111-1307 USA
+
+ Notifications
+
+ LXC supports a notification system that allows the container
+ to notify LXC when it has finished booting. If enabled, the
+ container has the status of BOOTING when control is given to
+ the init process and the container must notify lxc once the
+ boot sequence has completed. Since this requires intervention
+ from the inside of the container, it is disabled by default.
+
+
+ This system allows the administrator to distinguish from the
+ outside a container is that fully started (RUNNING) from a
+ container that is still in the process of initialization
+ (BOOTING).
+
+
+ The system is kept simple: LXC bind-mounts a UNIX socket into
+ the container (by default to
+ /dev/lxc-notify) and the container can
+ inform LXC via a very simple protocol. The following piece of
+ code may be used for the notification:
+
+
+ echo status RUNNING | perl -e 'use IO::Socket;
+ $client = IO::Socket::UNIX->new(
+ Peer => "/dev/lxc-notify",
+ Type => SOCK_STREAM,
+ Timeout => 10
+ );
+ while (<>) {
+ print $client $_;
+ }'
+
+
+ The following options control notifications:
+
+
+
+
+ lxc.notification.enabled
+
+
+
+ Whether the notification system is enabled. The values
+ true, yes,
+ on and 1 indicate that
+ it should be enabled, any other value that it should be
+ disabled.
+
+
+
+
+
+ lxc.notification.path
+
+
+
Re: [lxc-devel] [PATCH] Add mechanism for container to notify host about end of boot
Quoting Stéphane Graber (stgra...@ubuntu.com): > On 12-09-19 10:12 AM, Christian Seiler wrote: > > Hi, > > > >>> - is now in /run instead of /dev > >> > >> I don't think that part's going to work... Most distros mount /run as > >> tmpfs at boot time which will hide anything you're putting in there > >> before boot. > > > > Hmmm, that is indeed a problem... Do you have any suggestions? Or > > should we just keep it in /dev for now (path is configurable anyway) > > and worry about this later? > > > > Christian > > I think /dev is the safest at the moment. Arguably it's wrong as it's > not an actual device node, but it's the only directory that lxc already > requires all distros not to mess with (or we wouldn't have working > console, tty, ...). What are some other alternatives? We could use some sysvipc mechanism - just have the container share the ipcns with the monitor. We could create a directory (default /container, specifiable in the container config) where such communication files will be mounted. Let the per-distro templates set up the location and the distro-userspace to work together. Others? -serge -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH 2/2] allow short -h and -n options to lxc-ps
On 12-09-18 03:58 PM, Dwight Engen wrote: > --- > doc/lxc-ps.sgml.in |4 ++-- > src/lxc/lxc-ps.in |4 ++-- > 2 files changed, 4 insertions(+), 4 deletions(-) Looks good. Applied to staging. Thanks > diff --git a/doc/lxc-ps.sgml.in b/doc/lxc-ps.sgml.in > index f3275ae..401a17c 100644 > --- a/doc/lxc-ps.sgml.in > +++ b/doc/lxc-ps.sgml.in > @@ -51,7 +51,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA > 02111-1307 USA >lxc-ps >--name name >--lxc > - ps option > + -- ps option > > > > @@ -81,7 +81,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA > 02111-1307 USA > > > > - --name name > + -n, --name name > > > > diff --git a/src/lxc/lxc-ps.in b/src/lxc/lxc-ps.in > index a9923f0..4180aae 100644 > --- a/src/lxc/lxc-ps.in > +++ b/src/lxc/lxc-ps.in > @@ -74,9 +74,9 @@ containers="" > list_container_processes=0 > for i in "$@"; do > case $i in > - --help) > + -h|--help) > help; exit 1;; > - --name) > + -n|--name) > containers=$2; list_container_processes=1; shift 2;; > --lxc) > list_container_processes=1; shift;; > -- Stéphane Graber Ubuntu developer http://www.ubuntu.com signature.asc Description: OpenPGP digital signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] Add mechanism for container to notify host about end of boot
Hi, >> I think /dev is the safest at the moment. Arguably it's wrong as it's >> not an actual device node, but it's the only directory that lxc already >> requires all distros not to mess with (or we wouldn't have working >> console, tty, ...). > > What are some other alternatives? > > We could use some sysvipc mechanism - just have the container share > the ipcns with the monitor. The socket approach has the advantage that it can still be scripted from the shell - my guess would be that this becomes really hard with sysvipc. > We could create a directory (default /container, specifiable in > the container config) where such communication files will be > mounted. Let the per-distro templates set up the location and > the distro-userspace to work together. Since other people in this thread have expressed interest in having some general kind of directory to communicate with the container, perhaps this really is the best idea. Then this would consist of the following: 1) Create a directory that is shared between host and container More specifically: - mount a tmpfs with size=512k and nr_inodes=512 (should be MORE than sufficient to put a few sockets or similar there) to /var/lib/lxc/$name/interface (or wherever) - just before pivot_root: bind-mount it to /container or any place specified in the config 2) Create a lxc-specific socket inside /var/lib/lxc/$name/interface for status notifications when the mainloop is started. 3) Other applications may choose to put sockets there for their own purposes if they whish. The small tmpfs will make sure that the container can't do a disk space denial-of-service on the host. Thoughts? > Others? My guess is that other methods would certainly be possible but unless I'm missing something obvious, I don't think there's anything out there that isn't quite a bit more complicated than all the solutions discussed here. Regards, Christian -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
