Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-26 Thread Sarah Newman
On 11/26/2016 05:14 PM, Dario Faggioli wrote: > On Tue, 2016-11-22 at 22:40 +0100, Dario Faggioli wrote: >> On Tue, 2016-11-22 at 11:37 -0800, Sarah Newman wrote: >>> If you're saying not specifying "cpus=..." will keep libxl from >>> interfering with the Xens default allocation policy, then Xens >

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-26 Thread Dario Faggioli
On Tue, 2016-11-22 at 22:40 +0100, Dario Faggioli wrote: > On Tue, 2016-11-22 at 11:37 -0800, Sarah Newman wrote: > > If you're saying not specifying "cpus=..." will keep libxl from > > interfering with the Xens default allocation policy, then Xens > > default allocation > > policy no longer starts

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-22 Thread Dario Faggioli
On Tue, 2016-11-22 at 11:37 -0800, Sarah Newman wrote: > On 11/22/2016 10:46 AM, Dario Faggioli wrote: > > It's documented in here (although, the text can be improved a bit, > > I think)... look for 'cpus' and 'cpus_soft' within the page:  > > https://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-22 Thread Sarah Newman
On 11/22/2016 10:46 AM, Dario Faggioli wrote: > On Mon, 2016-11-21 at 13:06 -0800, Sarah Newman wrote: >> On 11/21/2016 11:37 AM, Sarah Newman wrote: >>> >>> If that's the reason not all the higher memory is being used first: is a >>> potential workaround to pin 64 bit domains to the second physi

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-22 Thread Dario Faggioli
On Mon, 2016-11-21 at 13:06 -0800, Sarah Newman wrote: > On 11/21/2016 11:37 AM, Sarah Newman wrote: > >  > > If that's the reason not all the higher memory is being used first: > > is a potential workaround to pin 64 bit domains to the second > > physical core on > > boot, and 32 bit domains to th

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-22 Thread Dario Faggioli
On Mon, 2016-11-21 at 11:37 -0800, Sarah Newman wrote: > On 11/21/2016 05:21 AM, Andrew Cooper wrote: > > I suspect that libxl's preference towards NUMA allocation of > > domains > > interferes with this, by adding a NUMA constraints to memory > > allocations > > for 64bit PV guests. > > I ran xl

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-21 Thread Sarah Newman
On 11/21/2016 11:37 AM, Sarah Newman wrote: > On 11/21/2016 05:21 AM, Andrew Cooper wrote: >> On 21/11/16 10:05, Jan Beulich wrote: > > Back in the xend days someone here had invented a (crude) mechanism > to set aside memory for 32-bit PV domains, but I don't think dealing with > this

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-21 Thread Sarah Newman
On 11/21/2016 05:21 AM, Andrew Cooper wrote: > On 21/11/16 10:05, Jan Beulich wrote: Back in the xend days someone here had invented a (crude) mechanism to set aside memory for 32-bit PV domains, but I don't think dealing with this situation in xl has ever seen any interest. >>> If

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-21 Thread Andrew Cooper
On 21/11/16 10:05, Jan Beulich wrote: On 21.11.16 at 10:45, wrote: >> On 11/21/2016 12:20 AM, Jan Beulich wrote: >> On 19.11.16 at 22:22, wrote: My current understanding is that on a server with more than 168GiB of memory, I should still be able to around 128GiB of 32-bit PV >>

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-21 Thread Jan Beulich
>>> On 21.11.16 at 10:45, wrote: > On 11/21/2016 12:20 AM, Jan Beulich wrote: > On 19.11.16 at 22:22, wrote: >>> My current understanding is that on a server with more than 168GiB >>> of memory, I should still be able to around 128GiB of 32-bit PV >>> domUs, regardless of what order the domUs

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-21 Thread Sarah Newman
On 11/21/2016 12:20 AM, Jan Beulich wrote: On 19.11.16 at 22:22, wrote: >> My current understanding is that on a server with more than 168GiB >> of memory, I should still be able to around 128GiB of 32-bit PV >> domUs, regardless of what order the domUs are started in. > > You don't clarify

Re: [Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-21 Thread Jan Beulich
>>> On 19.11.16 at 22:22, wrote: > Last night on a 288GiB server with less than 64GiB of 32 bit > domUs, we used the standard xendomains script which starts VMs > in alphabetical order. > > Some 32 bit domUs at the very end were unable to start. The > error message we received is the following: >

[Xen-devel] Can't always start 32 bit domains after 64 bit domains

2016-11-19 Thread Sarah Newman
Last night on a 288GiB server with less than 64GiB of 32 bit domUs, we used the standard xendomains script which starts VMs in alphabetical order. Some 32 bit domUs at the very end were unable to start. The error message we received is the following: xc: error: panic: xc_dom_x86.c:944: arch_setup