Hello, I tried by disabling mlockall and not do see long time for EAL init and is done quickly. Thanks for help.
ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach OVS logs 2020-07-29T09:48:00.233Z|00008|dpdk|INFO|Using DPDK 18.11.6 2020-07-29T09:48:00.233Z|00009|dpdk|INFO|DPDK Enabled - initializing... 2020-07-29T09:48:00.233Z|00010|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2020-07-29T09:48:00.233Z|00011|dpdk|INFO|IOMMU support for vhost-user-client disabled. 2020-07-29T09:48:00.233Z|00012|dpdk|INFO|POSTCOPY support for vhost-user-client disabled. 2020-07-29T09:48:00.233Z|00013|dpdk|INFO|Per port memory for DPDK devices disabled. 2020-07-29T09:48:00.233Z|00014|dpdk|INFO|EAL ARGS: ovs-vswitchd -w 0000:04:00.0,support-multi-driver=1 -w 0000:42:00.0,support-multi-driver=1 -c 0x1 --huge-dir /mnt/huge_ovs_2M --socket-mem 2048,2048 --socket-limit 2048,2048. 2020-07-29T09:48:00.237Z|00015|dpdk|INFO|EAL: Detected 40 lcore(s) 2020-07-29T09:48:00.237Z|00016|dpdk|INFO|EAL: Detected 2 NUMA nodes 2020-07-29T09:48:00.237Z|00017|dpdk|INFO|EAL: Multi-process socket /var/run/dpdk/rte/mp_socket 2020-07-29T09:48:00.280Z|00018|dpdk|INFO|EAL: 99 hugepages of size 1073741824 reserved, but no mounted hugetlbfs found for that size 2020-07-29T09:48:00.281Z|00019|dpdk|INFO|EAL: Probing VFIO support... 2020-07-29T09:48:00.281Z|00020|dpdk|INFO|EAL: VFIO support initialized 2020-07-29T09:48:01.696Z|00021|dpdk|INFO|EAL: PCI device 0000:04:00.0 on NUMA socket 0 2020-07-29T09:48:01.696Z|00022|dpdk|INFO|EAL: probe driver: 8086:154d net_ixgbe 2020-07-29T09:48:01.696Z|00023|dpdk|INFO|EAL: using IOMMU type 1 (Type 1) 2020-07-29T09:48:01.940Z|00024|dpdk|INFO|EAL: Ignore mapping IO port bar(2) 2020-07-29T09:48:02.285Z|00025|dpdk|INFO|EAL: PCI device 0000:42:00.0 on NUMA socket 1 2020-07-29T09:48:02.285Z|00026|dpdk|INFO|EAL: probe driver: 8086:154d net_ixgbe 2020-07-29T09:48:02.412Z|00027|dpdk|INFO|EAL: Ignore mapping IO port bar(2) 2020-07-29T09:48:02.757Z|00028|dpdk|INFO|DPDK Enabled - initialized Regards, Balaji -----Original Message----- From: Burakov, Anatoly <anatoly.bura...@intel.com> Sent: Wednesday, July 29, 2020 2:38 PM To: David Marchand <david.march...@redhat.com>; Balaji K N <balaji....@ericsson.com> Cc: dev@dpdk.org Subject: Re: [dpdk-dev] long initialization time for EAL init On 29-Jul-20 9:52 AM, David Marchand wrote: > On Wed, Jul 29, 2020 at 10:51 AM Balaji K N <balaji....@ericsson.com> wrote: >> We are using OVS 2.12.0 with DPDK 18.11.2 release on Ubuntu 16.04 LTS. We >> do observe long initialization time for EAL init ( ~10 seconds for EAL >> memory initialization). >> >> Below is our hardware and memory configuration >> Hardware : Dell Server >> Sockets(NUMA) : 2 >> CPUs : Each NUMA with 10 CPUs (Intel(R) Xeon(R) >> CPU E5-2660 v2 @ 2.20GHz) >> RAM : 128 GB >> Hugepages reserverd : 2048 huge pages of 2MB and 100 Huge pages of 1G >> (reserved during boot time). We are using only 2MB huge pages for OVS >> process. >> NIC : Ethernet 10G 2P X520 ( ixgbe driver ) >> >> Passing below EAL arguments with OVS process. >> 2020-07-23T09:46:58.878Z|00014|dpdk|INFO|EAL ARGS: ovs-vswitchd -w >> 0000:04:00.0,support-multi-driver=1 -w 0000:42:00.0,support-multi-driver=1 >> -c 0x1 --huge-dir /mnt/huge_ovs_2M --socket-mem 2048,2048 --socket-limit >> 2048,2048. >> >> We debugged further in DPDK and found out that mmap called in >> eal_get_virtual_area is taking more time ( rte_eal_init-> >> rte_eal_memory_init -> rte_eal_memseg_init -> memseg_primary_init -> >> alloc_va_space -> eal_get_virtual_area ->mmap ). Long initialization time is >> noticeable difference compared to prior release. Any pointers would be >> helpful. > > Did you try to start ovs with the --no-mlockall option? > > That's most likely the culprit. I believe the recent mlock/MPROT changes are going to be backported to 18.11? -- Thanks, Anatoly
