bhyve uses all available memory during IO-intensive operations

[Dustin Wenz]

I'm using chyves on FreeBSD 11.1 RELEASE to manage a few VMs (guest OS is also 
FreeBSD 11.1). Their sole purpose is to house some medium-sized Postgres 
databases (100-200GB). The host system has 64GB of real memory and 112GB of 
swap. I have configured each guest to only use 16GB of memory, yet while doing 
my initial database imports in the VMs, bhyve will quickly grow to use all 
available system memory and then be killed by the kernel:

        kernel: swap_pager: I/O error - pageout failed; blkno 1735,size 4096, 
error 12
        kernel: swap_pager: I/O error - pageout failed; blkno 1610,size 4096, 
error 12
        kernel: swap_pager: I/O error - pageout failed; blkno 1763,size 4096, 
error 12
        kernel: pid 41123 (bhyve), uid 0, was killed: out of swap space

The OOM condition seems related to doing moderate IO within the VM, though 
nothing within the VM itself shows high memory usage. This is the chyves config 
for one of them:

        bargs                      -A -H -P -S
        bhyve_disk_type            virtio-blk
        bhyve_net_type             virtio-net
        bhyveload_flags
        chyves_guest_version       0300
        cpu                        4
        creation                   Created on Mon Oct 23 16:17:04 CDT 2017 by 
chyves v0.2.0 2016/09/11 using __create()
        loader                     bhyveload
        net_ifaces                 tap51
        os                         default
        ram                        16G
        rcboot                     0
        revert_to_snapshot
        revert_to_snapshot_method  off
        serial                     nmdm51
        template                   no
        uuid                       8495a130-b837-11e7-b092-0025909a8b56


I've also tried using different bhyve_disk_types, with no improvement. How is 
it that bhyve can use far more memory that I'm specifying?

        - .Dustin

smime.p7s
Description: S/MIME cryptographic signature