date:20130908

Re: [lxc-devel] RFC: Device Namespaces

2013-09-08 Thread Amir Goldstein

On Fri, Sep 6, 2013 at 7:50 PM, Eric W. Biederman wrote:

> Oren Laadan  writes:
>
> > Hi Serge,
> >
> >
> > On Thu, Aug 22, 2013 at 2:21 PM, Serge Hallyn  >wrote:
> >
> >> Quoting Oren Laadan (or...@cellrox.com):
> >> > Hi everyone!
> >> >
> >> > We [1] have been working on bringing lightweight virtualization to
> >> > Linux-based mobile devices like Android (or other Linux-based devices
> >> with
> >> > diverse I/O) and want to share our solution: device namespaces.
> >> >
> >> > Imagine you could run several instances of your favorite mobile OS or
> >> other
> >> > distributions in isolated containers, each under the impression of
> having
> >> > exclusive access to device drivers; Interact and switch between them
> >> within
> >> > a blink, no flashing, no reboot.
> >> >
> >> > Device namespaces are an extension to existing Linux kernel namespaces
> >> that
> >> > brings lightweight virtualization to Linux-based end-user devices,
> >> > primarily mobile devices.
> >> > Device namespaces introduce a private and virtual namespace for device
> >> > drivers to create the illusion for a process group that it interacts
> >> > exclusively with a set of drivers. Device namespaces also introduce
> the
> >> > concepts of an “active” namespace with which a user interacts, vs
> >> > “non-active” namespaces that run in the background, and the ability to
> >> > switch between them.[2]
> >>
> >> Note that unless I'm misunderstanding what you're saying here, this is
> >> also what net_ns does.  A netns can exist with no processes so long as
> >> you've bound its /proc/$$/ns/net somewhere.  You can then re-enter that
> >> ns using ns_attach.  I haven't looked closely enough yet to see whether
> >> you should be (or are) using the same interface.
> >>
> >>
> > To illustrate the need for device namespaces, consider the use case of
> > running two containers of your favorite OS (say, Android), on a single
> > physical phone. As a user, you either work in one container, or in the
> > other, and you will want to be able to switch between them (just like
> with
> > apps on mobile devices: you interact with one application at a time, and
> > switch between them).
> >
> > See here for a demo of how it works:  http://vimeo.com/60113683
> >
> > To accomplish this, device namespaces solve two shortcomings of existing
> > namespaces:
> >
> > 1. A namespace for device drivers:  each (Android) container needs a
> > private view of all devices. This includes logical drivers, like binder
> (in
> > Android) but also loop device; and physical devices, like the framebuffer
> > and the touch-screen.
> >
> > In other words, device namespaces virtualize the _major/minor_ and the
> > _state_ of device drivers. With the exception of VFS, network, and PTY
> > (note: all three offer/are virtual devices), device drivers are otherwise
> > not isolated between containers.
> >
> > 2. A namespace for interactive scenarios:  a namespace can be "active" -
> it
> > has access to the hardware, e.g. display and touch-screen. This will be
> the
> > container with which the user is interacting right now. Otherwise a
> > namespace is "non-active" - it still runs in the background, but can
> > neither alter the display nor receive input from the touch-screen.
> > Switching to another container means a context switch in the relevant
> > drivers, so that they restore the state and now "obey" the other
> namespace.
> >
> > You can also think about the "active" namespace as foreground, and the
> > "non-active" as background, akin to foreground/background processes in a
> > terminal with job-control. Similar to how a terminal delivers input to
> the
> > foreground task only but not to the background tasks - this is enforced
> by
> > the new device namespace.
> >
> > More details on this use-case are in the wiki:
> > https://github.com/Cellrox/devns-patches/wiki/Thinvisor).
>
> I think this is going to take some talking, and looking at code.
>
>
Hi Eric,

If we can get people to take a quick look at the code before LPC
that could make the LPC discussions more effective.
Even looking at one of the subsystem patches can give a basic
idea of the work we have done:
https://github.com/Cellrox/linux/commits/devns-goldfish-3.4

I think you are talking about having wrappers around your devices so you
> can share.  Which is not the quite same problem the rest of us have been
> thinking of when talking about a device namespace.
>

We are interested in all problems related to virtualizated view of devices
inside a container, so let our work so far be a starting point to discuss
all of them.


>
> My first impression is that this is better solved with more appropriate
> abstractions in userspace or in the kernel.
>
> But we can talk at LPC and see what we can hash out.
>

Looking forward to that :-)

Amir.


>
> Eric
> ___
> Containers mailing list
> contain...@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/l

[lxc-devel] [RFC] [PATCH 0/6] Major cgroup logic rewrite

2013-09-08 Thread Christian Seiler

Hi all,

As discussed previously, I've now done a major rewrite of the entire
cgroup logic. There are now no assumptions made whatsoever when it
comes to the cgroup mount points, the kernel information will be used
to determine the proper locations for everything. (Only /proc is
assumed to be working properly.)

The patch series can also be found at my github under


I ended up almost completely rewriting cgroup.c to make it fit the new
scheme. For this reason, for the last patch for cgroup.[ch] it's
probably better to look at the new version and not the diff.

I did test the whole enchilada quite a bit, but I'm very sure that
there are a lot more bugs in there. What does work:

 - lxc-start (and -info, -stop)
 - Choosing $name-1, $name-2, ... as cgroup if $name is already used up
 - lxc-attach
 - lxc-freeze / lxc-unfreeze
 - lxc-stop -k on a frozen container
 - All subsystems mounted together in /sys/fs/cgroup directly
 - All subsystems mounted into different subdirectories beneath
   /sys/fs/cgroup
 - Although I didn't try actual nesting, on the systems with
   subsystems together in /sys/fs/cgroup I did:
  mkdir /sys/fs/cgroup/init /foo
  echo 1 > /sys/fs/cgroup/init/tasks
  mount --bind /sys/fs/cgroup/init /foo
  umount /sys/fs/cgroup
  mount --bind /foo /sys/fs/cgroup
  umount /foo
   And then lxc-start, which worked and correctly put the container
   in the /init/lxc/$name cgroup. This means that even with the current
   mountcgroups hook nesting should work (although as discussed before,
   the current mountcgroups logic breaks other software relying on
   cgroups [1])

Known issues:

 - Sometimes cgroup cleanup doesn't work 100%, sometimes cgroups are
   left behind. With the name collision avoidance this isn't a
   catastrophe, but it merits further investigation.
 - If cgroups.use in lxc.conf is set to something that expands to
   zero cgroups (because all selected controllers are not mounted,
   for example), currently it will fail at a weird place with a weird
   error message. This should either be caught (i.e. correct error
   message) or perhaps even allowed (i.e. no cgroups in that case)
 - Generally speaking, the global lxc.conf itself is not documented
   at all, currently.

Unknowns:

 - Bindings for anything other than Python, don't even know whether it
   compiles properly. (Python I didn't test either.)
 - Unit tests: They never worked on my system anyway, so I have no idea
   whether they still work.

Comments appreciated.

-- Christian

[1] I do want to get to the mountcgroups hook, just not today. ;-)


--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel

[lxc-devel] [PATCH 1/6] global config: Unify parsing, add additional checks

2013-09-08 Thread Christian Seiler

Instead of duplicating the code for parsing the global config file for
each option, write one main function, lxc_global_config_value, that
does the parsing for an arbitrary option name and just call that
function from the existing ones.

Signed-off-by: Christian Seiler 
---
 src/lxc/utils.c |  137 ---
 src/lxc/utils.h |6 ++-
 2 files changed, 63 insertions(+), 80 deletions(-)

diff --git a/src/lxc/utils.c b/src/lxc/utils.c
index 9a7a41d..fd892c1 100644
--- a/src/lxc/utils.c
+++ b/src/lxc/utils.c
@@ -198,7 +198,7 @@ extern int mkdir_p(const char *dir, mode_t mode)
return 0;
 }
 
-static char *copypath(char *p)
+static char *copy_global_config_value(char *p)
 {
int len = strlen(p);
char *retbuf;
@@ -216,116 +216,97 @@ static char *copypath(char *p)
return retbuf;
 }
 
-char *default_lxcpath;
 #define DEFAULT_VG "lxc"
-char *default_lvmvg;
 #define DEFAULT_ZFSROOT "lxc"
-char *default_zfsroot;
 
-const char *default_lvm_vg(void)
+const char *lxc_global_config_value(const char *option_name)
 {
-   char buf[1024], *p;
-   FILE *fin;
-
-   if (default_lvmvg)
-   return default_lvmvg;
+   static const char *options[][2] = {
+   { "lvm_vg",  DEFAULT_VG  },
+   { "zfsroot", DEFAULT_ZFSROOT },
+   { "lxcpath", LXCPATH },
+   { NULL, NULL },
+   };
+   static const char *values[sizeof(options) / sizeof(options[0])] = { 0 };
+   const char *(*ptr)[2];
+   size_t i;
+   char buf[1024], *p, *p2;
+   FILE *fin = NULL;
+
+   for (i = 0, ptr = options; (*ptr)[0]; ptr++, i++) {
+   if (!strcmp(option_name, (*ptr)[0]))
+   break;
+   }
+   if (!(*ptr)[0]) {
+   errno = EINVAL;
+   return NULL;
+   }
+   if (values[i])
+   return values[i];
 
fin = fopen(LXC_GLOBAL_CONF, "r");
if (fin) {
while (fgets(buf, 1024, fin)) {
if (buf[0] == '#')
continue;
-   p = strstr(buf, "lvm_vg");
+   p = strstr(buf, option_name);
if (!p)
continue;
+   /* see if there was just white space in front
+* of the option name
+*/
+   for (p2 = buf; p2 < p; p2++) {
+   if (*p2 != ' ' && *p2 != '\t')
+   break;
+   }
+   if (p2 < p)
+   continue;
p = strchr(p, '=');
if (!p)
continue;
+   /* see if there was just white space after
+* the option name
+*/
+   for (p2 += strlen(option_name); p2 < p; p2++) {
+   if (*p2 != ' ' && *p2 != '\t')
+   break;
+   }
+   if (p2 < p)
+   continue;
p++;
while (*p && (*p == ' ' || *p == '\t')) p++;
if (!*p)
continue;
-   default_lvmvg = copypath(p);
+   values[i] = copy_global_config_value(p);
goto out;
}
}
-   default_lvmvg = DEFAULT_VG;
+   /* could not find value, use default */
+   values[i] = (*ptr)[1];
+   /* special case: if default value is NULL,
+* and there is no config, don't view that
+* as an error... */
+   if (!values[i])
+   errno = 0;
 
 out:
if (fin)
fclose(fin);
-   return default_lvmvg;
+   return values[i];
 }
 
-const char *default_zfs_root(void)
+const char *default_lvm_vg(void)
 {
-   char buf[1024], *p;
-   FILE *fin;
-
-   if (default_zfsroot)
-   return default_zfsroot;
-
-   fin = fopen(LXC_GLOBAL_CONF, "r");
-   if (fin) {
-   while (fgets(buf, 1024, fin)) {
-   if (buf[0] == '#')
-   continue;
-   p = strstr(buf, "zfsroot");
-   if (!p)
-   continue;
-   p = strchr(p, '=');
-   if (!p)
-   continue;
-   p++;
-   while (*p && (*p == ' ' || *p == '\t')) p++;
-   if (!*p)
-   continue;
-   default_zfsroot = copypath(p);
-   goto out;
-   }
-   }
-   default_zfsr

[lxc-devel] [PATCH 2/6] Add cgroup.pattern global configuration option

2013-09-08 Thread Christian Seiler

Signed-off-by: Christian Seiler 
---
 configure.ac|7 +++
 src/lxc/Makefile.am |3 ++-
 src/lxc/utils.c |1 +
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 4eaf329..f0f417c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -260,6 +260,12 @@ AC_ARG_WITH([rootfs-path],
[--with-rootfs-path=dir],
[lxc rootfs mount point]
)], [], [with_rootfs_path=['${libdir}/lxc/rootfs']])
+# cgroup pattern specification
+AC_ARG_WITH([cgroup-pattern],
+   [AC_HELP_STRING(
+   [--with-cgroup-pattern=pattern],
+   [pattern for container cgroups]
+   )], [], [with_cgroup_pattern=['/lxc/%n']])
 
 # Container log path.  By default, use $lxcpath.
 AC_MSG_CHECKING([Whether to place logfiles in container config path])
@@ -303,6 +309,7 @@ AS_AC_EXPAND(LXCTEMPLATEDIR, "$datadir/lxc/templates")
 AS_AC_EXPAND(LXCHOOKDIR, "$datadir/lxc/hooks")
 AS_AC_EXPAND(LXCINITDIR, "$libexecdir")
 AS_AC_EXPAND(LOGPATH, "$with_log_path")
+AC_SUBST(DEFAULT_CGROUP_PATTERN, ["$with_cgroup_pattern"])
 
 # Check for some standard kernel headers
 AC_CHECK_HEADERS([linux/unistd.h linux/netlink.h linux/genetlink.h],
diff --git a/src/lxc/Makefile.am b/src/lxc/Makefile.am
index da5ff15..8ee279f 100644
--- a/src/lxc/Makefile.am
+++ b/src/lxc/Makefile.am
@@ -100,7 +100,8 @@ AM_CFLAGS=-I$(top_srcdir)/src \
-DLOGPATH=\"$(LOGPATH)\" \
-DLXC_DEFAULT_CONFIG=\"$(LXC_DEFAULT_CONFIG)\" \
-DLXC_USERNIC_DB=\"$(LXC_USERNIC_DB)\" \
-   -DLXC_USERNIC_CONF=\"$(LXC_USERNIC_CONF)\"
+   -DLXC_USERNIC_CONF=\"$(LXC_USERNIC_CONF)\" \
+   -DDEFAULT_CGROUP_PATTERN=\"$(DEFAULT_CGROUP_PATTERN)\"
 
 if ENABLE_APPARMOR
 AM_CFLAGS += -DHAVE_APPARMOR
diff --git a/src/lxc/utils.c b/src/lxc/utils.c
index fd892c1..36d80b9 100644
--- a/src/lxc/utils.c
+++ b/src/lxc/utils.c
@@ -225,6 +225,7 @@ const char *lxc_global_config_value(const char *option_name)
{ "lvm_vg",  DEFAULT_VG  },
{ "zfsroot", DEFAULT_ZFSROOT },
{ "lxcpath", LXCPATH },
+   { "cgroup.pattern",  DEFAULT_CGROUP_PATTERN },
{ NULL, NULL },
};
static const char *values[sizeof(options) / sizeof(options[0])] = { 0 };
-- 
1.7.10.4


--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel

[lxc-devel] [PATCH 3/6] Add fopen_cloexec function to emulate 'e' mode

2013-09-08 Thread Christian Seiler

Newer glibc versions (that we can't require) allow for an additional
letter 'e' in the fopen mode that will cause the file to be opened with
the O_CLOEXEC flag, so that it will be closed if the program exec()s
away. This is important because if liblxc is used in a multithreaded
program, another thread might want to run a program. This options
prevents the leakage of file descriptors from LXC. This patch adds an
emulation for that that uses the open(2) syscall and fdopen(3). At some
later point in time, it may be dropped against fopen(..., "...e").

This commit also converts all fopen() calls in utils.c (where the
function is added) to fopen_cloexec(). Subsequently, other calls to
fopen() and open() should also be adapted.

Signed-off-by: Christian Seiler 
---
 src/lxc/utils.c |   50 --
 src/lxc/utils.h |3 +++
 2 files changed, 51 insertions(+), 2 deletions(-)

diff --git a/src/lxc/utils.c b/src/lxc/utils.c
index 36d80b9..b188c47 100644
--- a/src/lxc/utils.c
+++ b/src/lxc/utils.c
@@ -245,7 +245,7 @@ const char *lxc_global_config_value(const char *option_name)
if (values[i])
return values[i];
 
-   fin = fopen(LXC_GLOBAL_CONF, "r");
+   fin = fopen_cloexec(LXC_GLOBAL_CONF, "r");
if (fin) {
while (fgets(buf, 1024, fin)) {
if (buf[0] == '#')
@@ -391,7 +391,7 @@ int sha1sum_file(char *fnam, unsigned char *digest)
 
if (!fnam)
return -1;
-   if ((f = fopen(fnam, "r")) < 0) {
+   if ((f = fopen_cloexec(fnam, "r")) < 0) {
SYSERROR("Error opening template");
return -1;
}
@@ -477,3 +477,49 @@ const char** lxc_va_arg_list_to_argv_const(va_list ap, 
size_t skip)
 {
return (const char**)lxc_va_arg_list_to_argv(ap, skip, 0);
 }
+
+FILE *fopen_cloexec(const char *path, const char *mode)
+{
+   int open_mode = 0;
+   int step = 0;
+   int fd;
+   int saved_errno = 0;
+   FILE *ret;
+
+   if (!strncmp(mode, "r+", 2)) {
+   open_mode = O_RDWR;
+   step = 2;
+   } else if (!strncmp(mode, "r", 1)) {
+   open_mode = O_RDONLY;
+   step = 1;
+   } else if (!strncmp(mode, "w+", 2)) {
+   open_mode = O_RDWR | O_TRUNC | O_CREAT;
+   step = 2;
+   } else if (!strncmp(mode, "w", 1)) {
+   open_mode = O_WRONLY | O_TRUNC | O_CREAT;
+   step = 1;
+   } else if (!strncmp(mode, "a+", 2)) {
+   open_mode = O_RDWR | O_CREAT | O_APPEND;
+   step = 2;
+   } else if (!strncmp(mode, "a", 1)) {
+   open_mode = O_WRONLY | O_CREAT | O_APPEND;
+   step = 1;
+   }
+   for (; mode[step]; step++)
+   if (mode[step] == 'x')
+   open_mode |= O_EXCL;
+   open_mode |= O_CLOEXEC;
+
+   fd = (open_mode & O_CREAT) ?
+   open(path, open_mode, 0666) :
+   open(path, open_mode);
+   if (fd < 0)
+   return NULL;
+
+   ret = fdopen(fd, mode);
+   saved_errno = errno;
+   if (!ret)
+   close(fd);
+   errno = saved_errno;
+   return ret;
+}
diff --git a/src/lxc/utils.h b/src/lxc/utils.h
index 0ad9505..b79be44 100644
--- a/src/lxc/utils.h
+++ b/src/lxc/utils.h
@@ -148,6 +148,9 @@ static inline int signalfd(int fd, const sigset_t *mask, 
int flags)
 }
 #endif
 
+/* open a file with O_CLOEXEC */
+FILE *fopen_cloexec(const char *path, const char *mode);
+
 
 /**
  * BUILD_BUG_ON - break compile if a condition is true.
-- 
1.7.10.4


--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel

[lxc-devel] [PATCH 4/6] utils: Add string and array utility functions

2013-09-08 Thread Christian Seiler

Adds a few useful string and array manipulation functions to utils.[ch]

Signed-off-by: Christian Seiler 
---
 src/lxc/utils.c |  284 +++
 src/lxc/utils.h |   32 +++
 2 files changed, 316 insertions(+)

diff --git a/src/lxc/utils.c b/src/lxc/utils.c
index b188c47..dc98443 100644
--- a/src/lxc/utils.c
+++ b/src/lxc/utils.c
@@ -37,6 +37,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "utils.h"
 #include "log.h"
@@ -523,3 +524,286 @@ FILE *fopen_cloexec(const char *path, const char *mode)
errno = saved_errno;
return ret;
 }
+
+char *lxc_string_replace(const char *needle, const char *replacement, const 
char *haystack)
+{
+   ssize_t len = -1, saved_len = -1;
+   char *result = NULL;
+   size_t replacement_len = strlen(replacement);
+   size_t needle_len = strlen(needle);
+
+   /* should be executed exactly twice */
+   while (len == -1 || result == NULL) {
+   char *p;
+   char *last_p;
+   ssize_t part_len;
+
+   if (len != -1) {
+   result = calloc(1, len + 1);
+   if (!result)
+   return NULL;
+   saved_len = len;
+   }
+
+   len = 0;
+
+   for (last_p = (char *)haystack, p = strstr(last_p, needle); p; 
last_p = p, p = strstr(last_p, needle)) {
+   part_len = (ssize_t)(p - last_p);
+   if (result && part_len > 0)
+   memcpy(&result[len], last_p, part_len);
+   len += part_len;
+   if (result && replacement_len > 0)
+   memcpy(&result[len], replacement, 
replacement_len);
+   len += replacement_len;
+   p += needle_len;
+   }
+   part_len = strlen(last_p);
+   if (result && part_len > 0)
+   memcpy(&result[len], last_p, part_len);
+   len += part_len;
+   }
+
+   /* make sure we did the same thing twice,
+* once for calculating length, the other
+* time for copying data */
+   assert(saved_len == len);
+   /* make sure we didn't overwrite any buffer,
+* due to calloc the string should be 0-terminated */
+   assert(result[len] == '\0');
+
+   return result;
+}
+
+bool lxc_string_in_array(const char *needle, const char **haystack)
+{
+   for (; haystack && *haystack; haystack++)
+   if (!strcmp(needle, *haystack))
+   return true;
+   return false;
+}
+
+char *lxc_string_join(const char *sep, const char **parts, bool use_as_prefix)
+{
+   char *result;
+   char **p;
+   size_t sep_len = strlen(sep);
+   size_t result_len = use_as_prefix * sep_len;
+
+   /* calculate new string length */
+   for (p = (char **)parts; *p; p++)
+   result_len += (p > (char **)parts) * sep_len + strlen(*p);
+
+   result = calloc(result_len + 1, 1);
+   if (!result)
+   return NULL;
+
+   if (use_as_prefix)
+   strcpy(result, sep);
+   for (p = (char **)parts; *p; p++) {
+   if (p > (char **)parts)
+   strcat(result, sep);
+   strcat(result, *p);
+   }
+
+   return result;
+}
+
+char **lxc_normalize_path(const char *path)
+{
+   char **components;
+   char **p;
+   size_t components_len = 0;
+   size_t pos = 0;
+
+   components = lxc_string_split(path, '/');
+   if (!components)
+   return NULL;
+   for (p = components; *p; p++)
+   components_len++;
+
+   /* resolve '.' and '..' */
+   for (pos = 0; pos < components_len; ) {
+   if (!strcmp(components[pos], ".") || (!strcmp(components[pos], 
"..") && pos == 0)) {
+   /* eat this element */
+   free(components[pos]);
+   memmove(&components[pos], &components[pos+1], 
sizeof(char *) * (components_len - pos));
+   components_len--;
+   } else if (!strcmp(components[pos], "..")) {
+   /* eat this and the previous element */
+   free(components[pos - 1]);
+   free(components[pos]);
+   memmove(&components[pos-1], &components[pos+1], 
sizeof(char *) * (components_len - pos));
+   components_len -= 2;
+   pos--;
+   } else {
+   pos++;
+   }
+   }
+
+   return components;
+}
+
+bool lxc_string_in_list(const char *needle, const char *haystack, char _sep)
+{
+   char *token, *str, *saveptr = NULL;
+   char sep[2] = { _sep, '\0' };
+
+   if (!haystack || !needle)
+   return 0;
+
+   str = alloca(strlen(hays

[lxc-devel] [PATCH 5/6] utils: Add utility functions that write/read to entire files

2013-09-08 Thread Christian Seiler

Signed-off-by: Christian Seiler 
---
 src/lxc/utils.c |   91 +++
 src/lxc/utils.h |5 +++
 2 files changed, 96 insertions(+)

diff --git a/src/lxc/utils.c b/src/lxc/utils.c
index dc98443..02336d7 100644
--- a/src/lxc/utils.c
+++ b/src/lxc/utils.c
@@ -39,6 +39,12 @@
 #include 
 #include 
 
+#ifndef HAVE_GETLINE
+#ifdef HAVE_FGETLN
+#include <../include/getline.h>
+#endif
+#endif
+
 #include "utils.h"
 #include "log.h"
 
@@ -807,3 +813,88 @@ void **lxc_dup_array(void **array, lxc_dup_fn 
element_dup_fn, lxc_free_fn elemen
 
return result;
 }
+
+int lxc_write_to_file(const char *filename, const void* buf, size_t count, 
bool add_newline)
+{
+   int fd, saved_errno;
+   ssize_t ret;
+
+   fd = open(filename, O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, 0666);
+   if (fd < 0)
+   return -1;
+   ret = lxc_write_nointr(fd, buf, count);
+   if (ret < 0)
+   goto out_error; 
+   if ((size_t)ret != count)
+   goto out_error;
+   if (add_newline) {
+   ret = lxc_write_nointr(fd, "\n", 1);
+   if (ret != 1)
+   goto out_error;
+   }
+   close(fd);
+   return 0;
+
+out_error:
+   saved_errno = errno;
+   close(fd);
+   errno = saved_errno;
+   return -1;
+}
+
+int lxc_read_from_file(const char *filename, void* buf, size_t count)
+{
+   int fd = -1, saved_errno;
+   ssize_t ret;
+
+   fd = open(filename, O_RDONLY | O_CLOEXEC);
+   if (fd < 0)
+   return -1;
+
+   if (!buf || !count) {
+   char buf2[100];
+   size_t count2 = 0;
+   while ((ret = read(fd, buf2, 100)) > 0)
+   count2 += ret;
+   if (ret >= 0)
+   ret = count2;
+   } else {
+   memset(buf, 0, count);
+   ret = read(fd, buf, count);
+   }
+
+   if (ret < 0)
+   ERROR("read %s: %s", filename, strerror(errno));
+
+   saved_errno = errno;
+   close(fd);
+   errno = saved_errno;
+   return ret;
+}
+
+char *lxc_read_line_from_file(const char *filename)
+{
+   FILE *f;
+   char *line = NULL;
+   int saved_errno;
+   size_t sz = 0;
+
+   f = fopen_cloexec(filename, "r");
+   if (!f)
+   return NULL;
+
+   if (getline(&line, &sz, f) == -1) {
+   saved_errno = errno;
+   fclose(f);
+   errno = saved_errno;
+   return NULL;
+   }
+
+   fclose(f);
+
+   /* trim line, if necessary */
+   if (strlen(line) > 0 && line[strlen(line) - 1] == '\n')
+   line[strlen(line) - 1] = '\0';
+
+   return line;
+}
diff --git a/src/lxc/utils.h b/src/lxc/utils.h
index 7261846..4c2ab29 100644
--- a/src/lxc/utils.h
+++ b/src/lxc/utils.h
@@ -194,6 +194,11 @@ extern ssize_t lxc_read_nointr_expect(int fd, void* buf, 
size_t count, const voi
 extern int sha1sum_file(char *fnam, unsigned char *md_value);
 #endif
 
+/* read and write whole files */
+extern int lxc_write_to_file(const char *filename, const void* buf, size_t 
count, bool add_newline);
+extern int lxc_read_from_file(const char *filename, void* buf, size_t count);
+extern char *lxc_read_line_from_file(const char *filename);
+
 /* convert variadic argument lists to arrays (for execl type argument lists) */
 extern char** lxc_va_arg_list_to_argv(va_list ap, size_t skip, int do_strdup);
 extern const char** lxc_va_arg_list_to_argv_const(va_list ap, size_t skip);
-- 
1.7.10.4


--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel

Re: [lxc-devel] [Lxc-users] Working LXC templates?

2013-09-08 Thread Tony Su

After putting some thought into this,
IMO LXC badly needs a universal tool with the following features

- A single script should be used to run on  HostOS, creating
 supported Container OS. Although this would make the script
extremely large, IMO it would actually be easier to maintain in the
long run. Unknown at this point is how long it would take to load, but
once loaded I would expect the script to run very fast.

- Flexible enough to support any packaging system, and perhaps even
non-package systems if someone should present a strong case or is
willing to put in the work

- Use the available HostOS distro packages whenever possible. Do not
require packages from other distros or cross-platform packages. IMO
this means less reliance on supporting non-mainline packages, keeps
package and functionality support simple, understandable and
responsibility clear.

- The script should be clearly modular, with sections clearly marked
for the functionality in that section.

- The script as currently envisioned should be a single file to reduce
missing "script dependencies." Any calls should generally be only to
internal functions or to mainstream packages provided by the distro.
Separate custom scripts should be avoided because otherwise the
dependency script(s) may be hard to locate.

To this end,
I've done the intial baby steps towards architecting a new universal
lxc-create template script that should have all the above features and
likely within a week or so post an early alpha on my github for
anyonne to criticize/contribute/modify.

In the meantime, I would be interested to know if anyone is
successfully doing what the title of this message thread describes...
Creating a Container with a different GuestOS than the HostOS, and if
you could either provide a link or description to where the template
script can be found (plus, as  Nathaniel Copa described anything
special and extraordinary you had to do).

It would be done a lot faster except that this is being done in my spare time.
Now, if someone who deals with LXC a lot wanted to hire me to do this
and maybe more... :)

Tony



On Wed, Sep 4, 2013 at 10:52 AM, Natanael Copa  wrote:
> On Wed, 04 Sep 2013 09:40:49 -0400
> "Michael H. Warfield"  wrote:
>
>> I do think it is an issue with the whole "distribution agnostic
>> template" problem that may require some help from the distros or some
>> innovative ideas of how we can bootstrap distros using distro agnostic
>> tools (like stone knives and bear skins style install of the rootfs
>> using nothing more than tar, gzip, gpg, and curl or wget).
>
> This would be very nice. I have not had success with any templates
> except the debian on Alpine Linux. Debian works because we build a
> debootstrap package. Ubuntu template did not work because it uses
> 'arch' command which we don't have. (ok, should be trivial to implement
> if we want it bad enough - and I haven't tested current git templates)
>
> However, the alpine template in current git should work on any distro.
>
> Here is what we do:
> * download static apk-tools (package manager) and the package with the
>   public keys used for package signature checking.
>
> * unpack the the package manager and public keys package with tar.  The
>   package format is basically .tar.gz with some files in the beginning
>   with metadata, so the .apk files can be extracted with tar -zx.
>
> * verify that the public keys are unmodified against a sha256 sum that
>   is embedded in the template script.
>
> * verify that the static binary is unmodified using the public key and
>   openssl. The apk-tools-static package includes a signature for the
>   static binary.
>
> * use the verified static package manager to install a rootfs. The
>   package manager will use the previously downloaded pub keys.
>
> This should work on any x86/x86_64 distro with tar, gzip, openssl and
> wget.
>
>
> -nc

--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel

Re: [lxc-devel] [Lxc-users] Working LXC templates?

2013-09-08 Thread Michael H. Warfield

With all due respect...

On Sun, 2013-09-08 at 16:08 -0700, Tony Su wrote: 
> After putting some thought into this,
> IMO LXC badly needs a universal tool with the following features
> 
> - A single script should be used to run on  HostOS, creating
>  supported Container OS. Although this would make the script
> extremely large, IMO it would actually be easier to maintain in the
> long run.

Actually, no.  From my experience (30+ years in software development),
it would turn into a morass.

The problem here is that the maintainer(s) would then need to understand
how each and every distribution is installed and how it would be
installed on each and every distribution.  It would distill the worse of
all the problems we have now in the templates into one great big dung
pile.  It would rapidly become unmaintainable.  The "extremely large" is
the red letter warning that it will become unmaintainable as fewer and
fewer people understand what this great big huge blob does.

I've written perl scripts that count 3K+ lines long and have to have
over 50% of the lines be comments just so people can read it (ok - it's
perl, what can I say).  The template system with small, individually
maintained, distribution templates are vastly more maintainable.  I'm
pretty good with the Fedora stuff.  I'm not about to tell the
maintainers of the Ubuntu or OpenSuse stuff how to do their thing.  We
can formulate rules and test each other's templates, but we can not
write or maintain them.  But, we can be good at maintaining our own
templates.

> Unknown at this point is how long it would take to load, but
> once loaded I would expect the script to run very fast.

Extremely doubtful, in my experience, and the "run very fast" is not the
point and not the source of your problems.  From a systems analysis
basis, a function which is run rarely can run very inefficiently if it
does the right thing every time and is superior to something that runs
efficiently but missing way too many corner cases and is a complex
gemish-gamash to maintain because it's a great big hairball.  I don't
care about "run very fast".  I care about "does the right thing" in this
case.  Plus, if it's the Fedora create that's busted, you're limited in
the domain of script code you have to look at, even if you are
unfamiliar with the code over all.
> - Flexible enough to support any packaging system, and perhaps even
> non-package systems if someone should present a strong case or is
> willing to put in the work

It'll never happen.  It would take a huge design team, conversant in all
those packaging systems (I'm pretty damn good in yum and rpm, so-so in
pkg and apt-get, and absolutely suck at yast and zypper) and then it
would be "design by committee".  A committee is a life form with six or
more legs AND NO BRAIN.

> - Use the available HostOS distro packages whenever possible. Do not
> require packages from other distros or cross-platform packages. IMO
> this means less reliance on supporting non-mainline packages, keeps
> package and functionality support simple, understandable and
> responsibility clear.

Agreed.  But that is an argument for a modularized system with small,
distro specific, components melded with a small modularized system with
distro independent components.

BTW...  You think trying to install Fedora on OpenSuse is bad?  Try
installing OpenSuse on Fedora (or Ubuntu).  It's a disaster.  That
template won't even come off the dime (no offense to the author - we
just don't have the tools in our distros).  Here's what you get...

-- 
[root@forest mhw]# lxc-create -n OpenSuse -t opensuse

lxc-create: No config file specified, using the default config 
/etc/lxc/default.conf
/usr/share/lxc/templates/lxc-opensuse: line 369: type: zypper: not found
'zypper' command is missing
lxc-create: failed to execute template 'opensuse'
lxc-create: aborted
-- 

Epic fail.  There is no zypper on Fedora.

> - The script should be clearly modular, with sections clearly marked
> for the functionality in that section.

And that argues against one great big script (which would have to be
collectively maintained) and for smaller components (which could be
independently maintained).

> - The script as currently envisioned should be a single file to reduce
> missing "script dependencies." Any calls should generally be only to
> internal functions or to mainstream packages provided by the distro.
> Separate custom scripts should be avoided because otherwise the
> dependency script(s) may be hard to locate.

Totally disagree as my experience leads me to believe this ultimately
leads to bit-rot and deterioration in the maintainability where nobody
knows who wrote what line and why and nobody will touch it to avoid the
blame of breaking someone else's code.

Nope...  Not going to happen (IMNSHO).  I've run into this even with the
Fedora template.  Several times I've posted to the -devel list asking
who wrote this or that line and "what were you thinking" only to get no
answer.  N

Re: [lxc-devel] RFC: Device Namespaces

2013-09-08 Thread Eric W. Biederman

Amir Goldstein  writes:

> On Fri, Sep 6, 2013 at 7:50 PM, Eric W. Biederman
>  wrote:
>
> Hi Eric,
>
> If we can get people to take a quick look at the code before LPC
> that could make the LPC discussions more effective.
> Even looking at one of the subsystem patches can give a basic
> idea of the work we have done:
> https://github.com/Cellrox/linux/commits/devns-goldfish-3.4
>
> I think you are talking about having wrappers around your devices
> so you
> can share.  Which is not the quite same problem the rest of us
> have been
> thinking of when talking about a device namespace.
>
> We are interested in all problems related to virtualizated view of
> devices
> inside a container, so let our work so far be a starting point to
> discuss all of them.
>
> My first impression is that this is better solved with more
> appropriate
> abstractions in userspace or in the kernel.

As I read your code, you are solving the problem of one opener of a
device among a group of openers being able to access a device at a time.
Which leads to the question why can't the multiplexing happen in
userspace?

I think with your design it would not be possible to play a song in one
device namespace while doing work in the other.  As a security model
that isn't wrong but as someone trying to get work done that could be a
real pain.

The more common concern is to have devices we can use all of the time.

There may be a need for a device namespace and multiplexing access to
hardware devices makes that clearer.  So far nothing has risen to the
level of we actually need a device namespace to do X.  Especially in an
erra of hotplug and dynamic device numbers.

It is arguable that you could do your kind of device multiplexing with a
fuse device in userspace that implements your desired policy.

And policy is where cell situtation seems to fall down because it hard
codes one specific policy into the kernel, and a policy most situations
don't find useful.

Eric

--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel

Re: [lxc-devel] RFC: Device Namespaces

[lxc-devel] [RFC] [PATCH 0/6] Major cgroup logic rewrite

[lxc-devel] [PATCH 1/6] global config: Unify parsing, add additional checks

[lxc-devel] [PATCH 2/6] Add cgroup.pattern global configuration option

[lxc-devel] [PATCH 3/6] Add fopen_cloexec function to emulate 'e' mode

[lxc-devel] [PATCH 4/6] utils: Add string and array utility functions

[lxc-devel] [PATCH 5/6] utils: Add utility functions that write/read to entire files

Re: [lxc-devel] [Lxc-users] Working LXC templates?

Re: [lxc-devel] [Lxc-users] Working LXC templates?

Re: [lxc-devel] RFC: Device Namespaces

10 matches

Site Navigation

Mail list logo

Footer information