[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-10-04 Thread Launchpad Bug Tracker
This bug was fixed in the package libvirt - 0.9.2-4ubuntu15 --- libvirt (0.9.2-4ubuntu15) oneiric; urgency=low * Pull patches from upstream which prevent a race between lxc driver and controller while a container is started, easily exposed by nova. (LP: #842845) -- Serge Ha

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-10-03 Thread Serge Hallyn
@Chuck, sorry! wrong bug. I'm aboug to assign 861656 to you however. ** Changed in: libvirt (Ubuntu Oneiric) Assignee: Chuck Short (zulcss) => Serge Hallyn (serge-hallyn) ** Patch removed: "debdiff solving the issue" https://bugs.launchpad.net/ubuntu/+source/nova/+bug/842845/+attachme

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-10-03 Thread Serge Hallyn
@Chuck, assigning this to you in the hopes that you can either push the debdiff or merge it with any changes you have remaining for nova. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title:

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-10-03 Thread Serge Hallyn
** Patch added: "debdiff solving the issue" https://bugs.launchpad.net/ubuntu/+source/nova/+bug/842845/+attachment/2504441/+files/nova-nbd.debdiff ** Changed in: libvirt (Ubuntu Oneiric) Assignee: Serge Hallyn (serge-hallyn) => Chuck Short (zulcss) -- You received this bug notification

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-10-03 Thread Serge Hallyn
libvirt with this debdiff attached passes qa-regression-testing -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problems starting multiple lxc instances concurrently To manage notificat

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-10-03 Thread Serge Hallyn
** Patch added: "Proposed debdiff solving the issue" https://bugs.launchpad.net/ubuntu/+source/nova/+bug/842845/+attachment/2503169/+files/libvirt-lxc-race.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launch

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-10-03 Thread Serge Hallyn
This is actually fixed upstream. I have a new independent patch (which can be found in the thread listed in comment #17) which fixes this, but it will be better if I can cherrypick the lxc patch from git. ** Changed in: nova (Ubuntu Oneiric) Status: In Progress => Invalid -- You received

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-29 Thread Kate Stewart
** Tags added: rls-mgr-o-tracking -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problems starting multiple lxc instances concurrently To manage notifications about this bug go to: htt

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-29 Thread Dave Walker
** Changed in: libvirt (Ubuntu Oneiric) Assignee: (unassigned) => Serge Hallyn (serge-hallyn) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problems starting multiple lxc instance

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-28 Thread Serge Hallyn
Sent the patch upstream. The tread can be followed here: https://www.redhat.com/archives/libvir-list/2011-September/msg01181.html -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problem

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-28 Thread Serge Hallyn
Actually this libvirt debdiff may be a better solution. I'm going to send it to upstream libvirt (cc: openstack) for comment. ** Patch added: "libvirt.debdiff" https://bugs.launchpad.net/ubuntu/+source/nova/+bug/842845/+attachment/2475829/+files/libvirt.debdiff -- You received this bug noti

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-28 Thread Serge Hallyn
** Changed in: libvirt (Ubuntu Oneiric) Status: Invalid => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problems starting multiple lxc instances concurrently To ma

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-27 Thread Serge Hallyn
In the libvirt logs, I see 21:20:43.846: 1205: debug : remoteSerializeError:132 : prog=536903814 ver=1 proc=16 type=1 serial=1004, msg=internal error Unable to get cgroup for instance-003d followed a bit later by 21:20:43.856: 8627: debug : virCgroupMakeGroup:511 : Make group /libvirt/lxc/i

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-27 Thread Serge Hallyn
Here is the relevant excerpt from nova-compute.log. note that the first line (at /usr/lib/python2.7/dist- packages/nova/virt/libvirt/connection.py:604)is two lines before the timer call which should wait until the container is booted. 2011-09-27 21:20:43,830 DEBUG nova.virt.libvirt_conn [-] insta

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-27 Thread Serge Hallyn
With the debdiff above, I can still very rarely reproduce this - it's the same cause - nova is calling get_info while libvirt is still setting the cgroups. The nova-compute.log shows the unhandled exception now at _get_power_state in nova/compute/manager.py. I'm not sure why it is getting to that

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-27 Thread Dave Walker
@hallyn, Can you propose your patch to upstream trunk please - then we'll look to cherry pick it back. Thanks :) ** Changed in: nova (Ubuntu Oneiric) Assignee: Chuck Short (zulcss) => Serge Hallyn (serge-hallyn) -- You received this bug notification because you are a member of Ubuntu Bugs,

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-27 Thread Ubuntu QA's Bug Bot
The attachment "nova.debdiff" of this bug report has been identified as being a patch in the form of a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. In the event that this is in fact not a patch you can resolve t

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-27 Thread Serge Hallyn
With this debdiff applied, the bug no longer appears on my systems. ** Patch added: "nova.debdiff" https://bugs.launchpad.net/ubuntu/+source/nova/+bug/842845/+attachment/2471726/+files/nova.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscr

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-27 Thread Serge Hallyn
** Changed in: nova (Ubuntu Oneiric) Assignee: (unassigned) => Chuck Short (zulcss) ** Changed in: libvirt (Ubuntu Oneiric) Assignee: Serge Hallyn (serge-hallyn) => (unassigned) ** Changed in: nova (Ubuntu Oneiric) Status: Incomplete => In Progress ** Changed in: libvirt (Ubuntu

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-27 Thread Serge Hallyn
I see - nova-compute checks too quickly whether the vm has powered on. It asks libvirt to look up the vm, but libvirt complains that it has not (yet) created the cgroups. Not sure yet whether this should be changed at libvirt or at nova's end. -- You received this bug notification because you ar

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-27 Thread Serge Hallyn
I'm able to reproduce this with 'euca-run-instances -n 4'. But the libvirt instances do in fact start. virsh -c lxc:/// list shows the, but euca-describe-instances shows them pending. nova-compute.log shows: 2011-09-27 16:06:23,110 ERROR nova.exception [-] Uncaught exception (nova.exception): T

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-23 Thread Serge Hallyn
** Changed in: libvirt (Ubuntu Oneiric) Assignee: (unassigned) => Serge Hallyn (serge-hallyn) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problems starting multiple lxc instance

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-23 Thread Dave Walker
** Changed in: nova (Ubuntu) Milestone: None => ubuntu-11.10 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problems starting multiple lxc instances concurrently To manage notifica

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-21 Thread Dave Walker
zul, is this still and issue and are we able to reproduce it? Is there a nova issue? ** Also affects: libvirt (Ubuntu Oneiric) Importance: High Status: Confirmed ** Also affects: nova (Ubuntu Oneiric) Importance: High Status: New ** Changed in: libvirt (Ubuntu Oneiric) M

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-20 Thread Dave Walker
** Tags added: server-o-rs -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problems starting multiple lxc instances concurrently To manage notifications about this bug go to: https://bu

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-15 Thread Chuck Short
I had a look at this and this caused by libvirt in src/lxc/lxc_driver.c: 545 if (virCgroupForDomain(driver->cgroup, vm->def->name, &cgroup, 0) != 0) { 546 lxcError(VIR_ERR_INTERNAL_ERROR, 547 _("Unable to get cgroup for %s"), vm->def->name); 548

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-13 Thread Serge Hallyn
Can't seem to reproduce this with a simple for c in o1 o2 o3 o4; do virsh -c lxc:// start $c done so I'll set up an openstack box to test. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/84284

Re: [Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-12 Thread James Page
On Mon, 2011-09-12 at 13:36 +, Serge Hallyn wrote: > You don't get this problem at all doing the same thing with kvm > containers, right? Right -- James Page Ubuntu Server Developer -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubunt

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-12 Thread Serge Hallyn
You don't get this problem at all doing the same thing with kvm containers, right? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problems starting multiple lxc instances concurrently

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-09 Thread Clint Byrum
This looks more like a problem with libvirt, as its an internal libvirt error.. (nova.exception): TRACE: File "/usr/lib/pymodules/python2.7/nova/virt/libvirt/connection.py", line 1168, in get_info (nova.exception): TRACE: (state, max_mem, mem, num_cpu, cpu_time) = virt_dom.info() (nova.exceptio

[Bug 842845] Re: problems starting multiple lxc instances concurrently

2011-09-06 Thread James Page
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/842845 Title: problems starting multiple lxc instances concurrently To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+sou