Hi Stanislav,

 

I repeated test with disabled api trace

 

vpp# show api trace-status

RX Trace disabled

TX Trace disabled

 

vpp# show threads

ID     Name                Type        LWP     Sched Policy (Priority)  lcore  
Core   Socket State

0      vpp_main                        178961  other (0)                n/a    
n/a    n/a

 

Before Test

 

vpp# show memory main-heap verbose

Thread 0 vpp_main

  base 0x7f5f6b4aa000, size 8g, locked, unmap-on-destroy, name 'main heap'

    page stats: page-size 4K, total 2097152, mapped 40974, not-mapped 1525072, 
unknown 531106

      numa 0: 18302 pages, 71.49m bytes

      numa 1: 22672 pages, 88.56m bytes

    total: 7.99G, used: 36.12M, free: 7.96G, trimmable: 7.96G

      free chunks 150 free fastbin blks 0

      max total allocated 7.99G

 

After several hours of execution

 

vpp# show memory main-heap verbose

Thread 0 vpp_main

  base 0x7f5f6b4aa000, size 8g, locked, unmap-on-destroy, name 'main heap'

    page stats: page-size 4K, total 2097152, mapped 563066, not-mapped 1002980, 
unknown 531106

      numa 0: 545742 pages, 2.08g bytes

      numa 1: 17324 pages, 67.67m bytes

    total: 7.99G, used: 1.04G, free: 6.96G, trimmable: 6.96G

      free chunks 1 free fastbin blks 0

      max total allocated 7.99G

 

Best Regards, 

Eugene

 

From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Stanislav Zaikin
Sent: Wednesday, February 22, 2023 11:46 AM
To: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] VPP Policer API Memory Leak

 

Hi Eugene,

 

Could you run again with disabled api trace, wait until let's say 1g is 
consumed and then show us the output of show memory main-heap verbose?

 

On Tue, 21 Feb 2023 at 20:13, <efimochki...@gmail.com 
<mailto:efimochki...@gmail.com> > wrote:

Hi Steven,

 

Thanks for response.

 

I added “nitems 65535” and repeated test. The main-heap usage still growing up.

Also I completely disabled the “api-trace” and nothing changed =(

 

Best Regards, 

Eugene

 

 

From: vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>  <vpp-dev@lists.fd.io 
<mailto:vpp-dev@lists.fd.io> > On Behalf Of steven luong via lists.fd.io 
<http://lists.fd.io> 
Sent: Tuesday, February 21, 2023 8:58 PM
To: vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> 
Subject: Re: [vpp-dev] VPP Policer API Memory Leak

 

I bet you didn’t limit the number of API trace entries. Try limit the number of 
API trace entries that VPP keeps with nitems and give it a reasonable number.

 

                api-trace {

                  on 

                nitems 65535

                }

 

Steven

 

From: <vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> > on behalf of 
"efimochki...@gmail.com <mailto:efimochki...@gmail.com> " 
<efimochki...@gmail.com <mailto:efimochki...@gmail.com> >
Reply-To: "vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> " 
<vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> >
Date: Tuesday, February 21, 2023 at 7:14 AM
To: "vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> " <vpp-dev@lists.fd.io 
<mailto:vpp-dev@lists.fd.io> >
Subject: [vpp-dev] VPP Policer API Memory Leak

 

Hi Dear Developers,

 

I am testing creating and deleting of policers and it looks that there is a 
memory leak

 

VPP Version: v22.10-release

 

 

My simple script:

 

#!/bin/env python

 

from vpp_papi import VPPApiClient

from vpp_papi import VppEnum

import os

import fnmatch

import sys

from time import sleep

 

vpp_json_dir = '/usr/share/vpp/api/'

 

# construct a list of all the json api files

 

jsonfiles = []

 

for root, dirnames, filenames in os.walk(vpp_json_dir):

  for filename in fnmatch.filter(filenames, '*.api.json'):

    jsonfiles.append(os.path.join(root, filename))

 

vpp = VPPApiClient(apifiles=jsonfiles, server_address='/run/vpp/api.sock')

vpp.connect("test-client")

 

r = vpp.api.show_version()

print('VPP version is %s' % r.version)

 

while True:

### Create 10 policers

  for i in range (10):

    name = "policer_" + str(i)

    policer_add_del = vpp.api.policer_add_del(is_add=True, name=name, 
cb=2500,cir=1000, eb=3000,eir=0,rate_type=0,round_type=1,type=1)

    print(policer_add_del)

### Delete 10 policers

  for i in range (10):

    name = "policer_" + str(i)

    policer_add_del = vpp.api.policer_add_del(is_add=False, name=name, 
cb=2500,cir=1000, eb=3000,eir=0,rate_type=0,round_type=1,type=1)

    print(policer_add_del)

 

The memory usage is growing permanently and very fast. It takes less than 10 
minutes to spend ~ 100Mb of main-heap.

 

vpp# show memory  main-heap

Thread 0 vpp_main

  base 0x7efb0a117000, size 8g, locked, unmap-on-destroy, traced, name 'main 
heap'

    page stats: page-size 4K, total 2097152, mapped 116134, not-mapped 1450398, 
unknown 530620

      numa 0: 115788 pages, 452.29m bytes

      numa 1: 346 pages, 1.35m bytes

    total: 7.99G, used: 188.26M, free: 7.82G, trimmable: 7.82G

 

  Bytes    Count     Sample   Traceback

  1774488    14781 0x7efb15d59570 _vec_alloc_internal + 0x6b

                                  vl_msg_api_trace + 0x4a4

                                  vl_msg_api_socket_handler + 0x10f

                                  vl_socket_process_api_msg + 0x1d

                                  0x7efd0c177171

                                  0x7efd0a588837

                                  0x7efd0a48d6a8

   291272        1 0x7efb15cf4190 _vec_realloc_internal + 0x89

                                  vl_msg_api_trace + 0x529

                                  vl_msg_api_socket_handler + 0x10f

                                  vl_socket_process_api_msg + 0x1d

                                  0x7efd0c177171

                                  0x7efd0a588837

                                  0x7efd0a48d6a8

   178928     7390 0x7efb15d595f0 _vec_alloc_internal + 0x6b

                                  va_format + 0x2318

                                  format + 0x83

                                  0x7efd0a896b91

                                  vl_msg_api_socket_handler + 0x226

                                  vl_socket_process_api_msg + 0x1d

                                  0x7efd0c177171

                                  0x7efd0a588837

                                  0x7efd0a48d6a8

    85800        1 0x7efb135ca840 _vec_realloc_internal + 0x89

                                  vl_socket_api_send + 0x720

                                  vl_api_sockclnt_create_t_handler + 0x2e2

                                  vl_msg_api_socket_handler + 0x226

                                  vl_socket_process_api_msg + 0x1d

                                  0x7efd0c177171

                                  0x7efd0a588837

                                  0x7efd0a48d6a8

     4104        1 0x7efb13dcf220 _vec_alloc_internal + 0x6b

                                  0x7efd0a5e0965

                                  0x7efd0a5f05c4

                                  0x7efd0a584978

                                  0x7efd0a5845f5

                                  0x7efd0a5f213b

                                  0x7efd0a48d6a8

     1920       16 0x7efb13e62a40 _vec_realloc_internal + 0x89

                                  0x7efd0a482d1d

                                  va_format + 0xf62

                                  format + 0x83

                                  va_format + 0x1041

                                  format + 0x83

                                  va_format + 0x1041

                                  vlib_log + 0x2c6

                                  0x7efb08b033aa

                                  0x7efb08b031c9

                                  0x7efb08b0cc6d

                                  0x7efb08b988ee

 

vpp# show memory main-heap verbose

Thread 0 vpp_main

  base 0x7efb0a117000, size 8g, locked, unmap-on-destroy, traced, name 'main 
heap'

    page stats: page-size 4K, total 2097152, mapped 170152, not-mapped 1396380, 
unknown 530620

      numa 0: 169806 pages, 663.30m bytes

      numa 1: 346 pages, 1.35m bytes

    total: 7.99G, used: 289.51M, free: 7.72G, trimmable: 7.71G

      free chunks 93 free fastbin blks 0

      max total allocated 7.99G

 

  Bytes    Count     Sample   Traceback

29822832   248408 0x7efb18bbfaf0 _vec_alloc_internal + 0x6b

                                  vl_msg_api_trace + 0x4a4

                                  vl_msg_api_socket_handler + 0x10f

                                  vl_socket_process_api_msg + 0x1d

                                  0x7efd0c177171

                                  0x7efd0a588837

                                  0x7efd0a48d6a8

22669680   822640 0x7efb19210510 _vec_alloc_internal + 0x6b

                                  va_format + 0x2318

                                  format + 0x83

                                  0x7efd0a896b91

                                  vl_msg_api_socket_handler + 0x226

                                  vl_socket_process_api_msg + 0x1d

                                  0x7efd0c177171

                                  0x7efd0a588837

                                  0x7efd0a48d6a8

  2212040        1 0x7efb17ee2220 _vec_realloc_internal + 0x89

                                  vl_msg_api_trace + 0x529

                                  vl_msg_api_socket_handler + 0x10f

                                  vl_socket_process_api_msg + 0x1d

                                  0x7efd0c177171

                                  0x7efd0a588837

                                  0x7efd0a48d6a8

    85800        1 0x7efb135ca840 _vec_realloc_internal + 0x89

                                  vl_socket_api_send + 0x720

                                  vl_api_sockclnt_create_t_handler + 0x2e2

                                  vl_msg_api_socket_handler + 0x226

                                  vl_socket_process_api_msg + 0x1d

                                  0x7efd0c177171

                                  0x7efd0a588837

                                  0x7efd0a48d6a8

 

Thanks in Advance

 

Best Regards, 

Eugene







-- 

Best regards
Stanislav Zaikin

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22623): https://lists.fd.io/g/vpp-dev/message/22623
Mute This Topic: https://lists.fd.io/mt/97110527/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Reply via email to