I ran that command on my own development host rather than in the Xen DDK. (I use i386 instead of x86-64 to put some diversity into development, since most developers use x86-64.) Rest assured that we do all of our XenServer builds in the Xen DDK that corresponds to the target XenServer target (and we always have don so).
32- versus 64-bit shouldn't affect where the best opportunities for reducing stack use lie. On Wed, Jul 09, 2014 at 03:12:14PM +0100, Anoob Soman wrote: > Hi Ben, > > From the objdump-sed (%esp) command I realised that you guys might > be using a XenServer DDK shipped with 6.2, which is 32bit. From > Creedence (next release) onwards dom0 is 64bit, and so are the > tools. > http://xenserver.org/open-source-virtualization-download/24-product/creedence/143-xs-2014-development-snapshots.html > > Thanks, > Anoob. > On 08/07/14 17:47, Ben Pfaff wrote: > >I guess that the biggest effect on stack size would be the flow table > >and in particular how much recursion flow processing causes. There are > >a few tests that force as-deep-as-possible recursion: > > > > AT_SETUP([ofproto-dpif - infinite resubmit]) > > AT_SETUP([ofproto-dpif - exponential resubmit chain]) > > > >I don't think that forcing all packets to userspace would have much of > >an effect. (The closest equivalent would be to disable megaflows, > >there's an "ovs-appctl" command for that, look in "ovs-appctl help".) > > > >Another hint toward maximum stack requirement is to look through the > >generated asm for stack usage, e.g.: > > > > objdump -dr vswitchd/ovs-vswitchd|sed -n > > 's/^.*sub.*$0x\([0-9a-f]\{1,\}\),%esp/\1/p'|sort|uniq|less > > > >which shows that we have at least one place where we allocate 327,788 > >bytes on the stack (!). I hope that is not in the flow processing path! > > > >On Tue, Jul 08, 2014 at 05:36:07PM +0100, Anoob Soman wrote: > >>I have been running tests with 1MB stack size and ovs-vswitchd seem > >>to hold pretty well. I will try to do some more experiments to find > >>out the max depth of the stack, but I am afraid this will totally > >>depend on the test I am running. Any suggestion on what sort of test > >>I should be running ? More over "force-miss-model" other-config is > >>missing from 2.1.x as there is no concept of facets. Is there way > >>that I can force all packets to be processed in userspace, other > >>than me doing "ovs-dpctl del-flows" periodically. > >> > >>Thanks, > >>Anoob. > >>On 08/07/14 17:15, Ben Pfaff wrote: > >>>On Tue, Jul 08, 2014 at 05:08:43PM +0100, Anoob Soman wrote: > >>>>Since openvswitch has moved to multi-threaded model, RSS usage of > >>>>ovs-vswitchd has increased quite significantly compared to the last > >>>>release we used (ovs-1.4.x). Part of the problem is using mlockall > >>>>(with MCL_CURRENT|MCL_FUTURE) on ovs-vswitchd, which causes every > >>>>pthreads stack's and heap's virtual address to locked to RAM. > >>>>ovs-vswitch (2.1.x) running on a 8 vCPU dom0 (10 pthreads) uses > >>>>around 89M of RSS (80MB just for stack), without any VMs running on > >>>>the host. One way to reduce RSS would be to reduce the number of > >>>>"n-handler-threads" and "n-revalidator-threads", but I am not sure > >>>>about the performance impact of having these thread numbers reduced. > >>>>I am wondering if the stack size of the pthreads can be reduce > >>>>(using pthread_attr_setstack). By default pthreads max stack size is > >>>>8MB and mlockall locks all of this 8MB into RAM. What could be > >>>>optimal stack size that can be used. > >>>I think it would be very reasonable to reduce the stack sizes, but I > >>>don't know the "correct" size off-hand. Since you're looking at the > >>>problem already, perhaps you should consider some experiments. > _______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
