> On Jan 31, 2021, at 6:13 PM, Elliott Mitchell <ehem+...@m5p.com> wrote: > > On Thu, Jan 28, 2021 at 10:42:27PM +0000, George Dunlap wrote: >> >>> On Jan 28, 2021, at 6:26 PM, Elliott Mitchell <ehem+...@m5p.com> wrote: >>> type = "hvm" >>> memory = 1024 >>> maxmem = 1073741824 >>> >>> I suspect maxmem > free Xen memory may be sufficient. The instances I >>> can be certain of have been maxmem = total host memory *7. >> >> Can you include your Xen version and dom0 command-line? > >> This is on staging-4.14 from a month or two ago (i.e., what I happened to >> have on a random test box), and `dom0_mem=1024M,max:1024M` in my >> command-line. That rune will give dom0 only 1GiB of RAM, but also prevent >> it from auto-ballooning down to free up memory for the guest. >> > > As this is a server, not a development target, Debian's build of 4.11 is > in use. Your domain 0 memory allocation is extremely generous compared > to mine. One thing which is on the command-line though is > "watchdog=true".
staging-4.14 is just the stable 4.14 branch which our CI loop tests before pushing to stable-4.14, which is essentially tagged 3 times a year for point releases. It’s quite stable. I’ll give 4.11 a try if I get a chance. It’s not clear from your response — are you allocating a fixed amount to dom0? How much is it? In fact, probably the simplest thing to do would be to attach the output of `xl info` and `xl dmesg`; that will save a lot of potential future back-and-forth. 1GiB isn’t particularly generous if you’re running a large number of guests. My understanding is that XenServer now defaults to 4GiB of RAM for dom0. > I've got 3 candidates which presently concern me:ble: > > 1> There is a limited range of maxmem values where this occurs. Perhaps > 1TB is too high on your machine for the problem to reproduce. As > previously stated my sample configuration has maxmem being roughly 7 > times actual machine memory. In fact I did a number of binary-search-style experiments to try to find out boundary behavior. I don’t think I did 7x memory, but I certainly did 2x or 3x host memory, and the exact number you gave that caused you problems. In all cases for me, it either worked or failed with a cryptic error message (the specific message depending on whether I had fixed dom0 memory or autoballooned memory). > 2> Between issuing the `xl create` command and the machine rebooting a > few moments of slow response have been observed. Perhaps the memory > allocator loop is hogging processor cores long enough for the watchdog to > trigger? I don’t know the balloon driver very well, but I’d hope it yielded pretty regularly. It seems more likely to me that your dom0 is swapping due to low memory / struggling with having to work with no file cache. Or the OOM killer is doing its calculation trying to figure out which process to shoot? > 3> Perhaps one of the patches on Debian broke things? This seems > unlikely since nearly all of Debian's patches are either strictly for > packaging or else picks from Xen's main branch, but this is certainly > possible. Indeed, I’d consider that unlikely. Some things I’d consider more likely to cause the difference: 1. The amount of host memory (my test box had only 6GiB) 2. The amount of memory assigned to dom0 3. The number of other VMs running in the background 4. A difference in the version of Linux (I’m also running Debian, but deban-testing) 5. A bug in 4.11 that was fixed by 4.14. If you’re already allocating a fixed amount of memory to dom0, but it’s significantly less than 1GiB, the first thing I’d try is increasing that to 1GiB. Also make sure that you’re specifying a ‘max’ for dom0 memory: If you simply put `dom0_mem=X`, dom0 will start with X amount of memory, but allocate enough frame tables such that it could balloon up to the full host memory if requested. (And frame tables are not free.) `dom0_mem=X,max=X` will cause dom0 to only make frame tables for X memory. (At least, so I’m guessing; I haven’t checked.) If that doesn’t work, please include the output of `xl info` and `xl dmesg`; that will give us a lot more information to work with. Peace, -George
