Hi Matthew, The patch (https://gerrit.fd.io/r/#/c/16287/) was intended to allocate crypto mem pool from DPDK, instead of from vPP. I guess you are using 2MB huge page, so you are experiencing out of memory with new patch created by Damjan. Please switch to 1GB, to see if it still happens.
Hi Damjan, As I understand, DPDK could dynamically allocate mem segment when huge page is 1GB and there is still free space. In this case the socket-mem is valid always. Therefore, please do not merge the patch (https://gerrit.fd.io/r/#/c/16543/), it won't work for 2MB case. Regards, Kingwel From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Damjan Marion via Lists.Fd.Io Sent: Friday, December 21, 2018 2:02 AM To: Matthew Smith <mgsm...@netgate.com> Cc: vpp-dev@lists.fd.io Subject: Re: [vpp-dev] dpdk: switch to in-memory mode, deprecate use of socket-mem On 20 Dec 2018, at 18:46, Matthew Smith <mgsm...@netgate.com<mailto:mgsm...@netgate.com>> wrote: Hi Damjan, There is a comment that says "preallocate at least 16MB of hugepages per socket, if more is needed it is up to consumer to preallocate more". What does a consumer need to do in order to preallocate more? This is just equivalent of "sysctl -w vm.nr_hugepages=X", if you set more with sysctl or kernel boot parameters, vpp will not try to change that. We can make this configurable from startup.conf, but in reality it is just convenience feature. I've recently had problems using AES-GCM with IPsec on a test system. Mempool allocation for the DPDK aesni_gcm crypto vdev fails during initialization. An error message like this is logged: vnet[4765]: crypto_create_session_drv_pool: failed to create session drv mempool I found that the DPDK memory allocation set with socket-mem is 64 MB by default and this is being exhausted by the various mempool allocations. This behavior seems to have started with change https://gerrit.fd.io/r/#/c/16287/ ("dpdk-ipsec-mempool: allocate from dpdk mem specified by socket-mem in startup.conf"). When I increase socket-mem in startup.conf to 128 MB, the configured crypto devices successfully initialize. you can see what exactly dpdk allocated with "show dpdk physmem". When support for socket-mem goes away, what will need to be done to ensure that enough memory is available for crypto devices? DPDK is supposed to dynamically alloc more memory if needed, Unless I missed something... Thanks, -Matt On Thu, Dec 20, 2018 at 3:55 AM Damjan Marion via Lists.Fd.Io<http://lists.fd.io/> <dmarion=me....@lists.fd.io<mailto:me....@lists.fd.io>> wrote: Regarding: https://gerrit.fd.io/r/#/c/16543/ This patch switches dpdk to new in-memory mode, and reduces dpdk memory footprint as pages are allocated dynamically on-demand. I tested on both Ubuntu and Centos 7.5 and everything looks good but will appreciate feedback from people using non-standard configs before it is merged... Thanks, -- Damjan -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11724): https://lists.fd.io/g/vpp-dev/message/11724 Mute This Topic: https://lists.fd.io/mt/28809973/675725 Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev%2bow...@lists.fd.io> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [mgsm...@netgate.com<mailto:mgsm...@netgate.com>] -=-=-=-=-=-=-=-=-=-=-=- -- Damjan
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11743): https://lists.fd.io/g/vpp-dev/message/11743 Mute This Topic: https://lists.fd.io/mt/28809973/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
