Looks like a simple and worthwhile optimization, merged. Thanks! ben
> -----Original Message----- > From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Velichkin > Viktor > Sent: Tuesday, May 31, 2022 22:47 > To: vpp-dev@lists.fd.io > Subject: [vpp-dev] papi performance improvement patch > > Hi! > > Can anyone please reveiw this patch <https://gerrit.fd.io/r/c/vpp/+/36308> > ? > > Explanation: > > For small amount of objects (for example short ACL) there is no problems > in python api, but if you trying to load thousands - papi works very slow. > Loading ACL with 256k entries could take forever. > > Here below is comparsion of uploading 128k entries ACL: > > - excuted command: > > r = vpp.api.acl_add_replace(acl_index=0xffffffff, > count=len(rules[:128000]) , r=rules[:128000]) > > > - without fix it took 121 seconds, and scales quadratically: > > 15488160 function calls (14592160 primitive calls) in > 121.454 seconds > > Ordered by: cumulative time > > ncalls tottime percall cumtime percall > filename:lineno(function) > 1 0.000 0.000 121.454 121.454 {built-in method > builtins.exec} > 1 0.022 0.022 121.454 121.454 <string>:1(<module>) > 1 0.000 0.000 121.432 121.432 > vpp_papi.py:120(__call__) > 1 0.000 0.000 121.432 121.432 vpp_papi.py:494(f) > 1 0.001 0.001 121.432 121.432 > vpp_papi.py:694(_call_vpp) > 896001/1 6.084 0.000 118.642 118.642 > vpp_serializer.py:627(pack) > 1 103.522 103.522 118.640 118.640 > vpp_serializer.py:298(pack) > 256000 0.469 0.000 9.855 0.000 > vpp_serializer.py:47(conversion_packer) > 256000 1.368 0.000 3.942 0.000 > vpp_serializer.py:472(pack) > 256008 1.338 0.000 3.349 0.000 {method 'format' of > 'str' objects} > 1 0.000 0.000 2.715 2.715 > vpp_papi.py:728(<listcomp>) > 256000 0.448 0.000 2.011 0.000 > ipaddress.py:610(__repr__) > 256000 0.435 0.000 1.563 0.000 > ipaddress.py:613(__str__) > 1792005 0.819 0.000 1.330 0.000 > vpp_serializer.py:111(pack) > 1152001 0.690 0.000 1.281 0.000 > vpp_serializer.py:37(conversion_required) > 256000 0.610 0.000 1.249 0.000 > vpp_format.py:134(<lambda>) > 256000 0.323 0.000 1.151 0.000 > vpp_serializer.py:521(pack) > 256000 0.215 0.000 1.077 0.000 > ipaddress.py:585(__str__) > 512000 0.475 0.000 0.897 0.000 > vpp_serializer.py:412(pack) > 256000 0.281 0.000 0.862 0.000 > ipaddress.py:1162(_string_from_ip_int) > 2304008 0.774 0.000 0.774 0.000 {built-in method > builtins.isinstance} > 256002 0.194 0.000 0.697 0.000 > __init__.py:1286(debug) > > > > - with proposed fix it took just 16.5 seconds, and what more important it > scales linearly (that means 256k will upload for ~ 32 seconds) > > 15488160 function calls (14592160 primitive calls) in > 16.516 seconds > > Ordered by: cumulative time > > ncalls tottime percall cumtime percall > filename:lineno(function) > 1 0.000 0.000 16.516 16.516 {built-in method > builtins.exec} > 1 0.012 0.012 16.516 16.516 <string>:1(<module>) > 1 0.000 0.000 16.504 16.504 > vpp_papi.py:120(__call__) > 1 0.000 0.000 16.504 16.504 vpp_papi.py:494(f) > 1 0.001 0.001 16.504 16.504 > vpp_papi.py:694(_call_vpp) > 896001/1 5.690 0.000 13.723 13.723 > vpp_serializer.py:627(pack) > 1 0.133 0.133 13.719 13.719 > vpp_serializer.py:298(pack) > 256000 0.474 0.000 9.162 0.000 > vpp_serializer.py:47(conversion_packer) > 256000 1.155 0.000 3.370 0.000 > vpp_serializer.py:472(pack) > 256008 1.162 0.000 3.161 0.000 {method 'format' of > 'str' objects} > 1 0.000 0.000 2.706 2.706 > vpp_papi.py:728(<listcomp>) > 256000 0.445 0.000 1.999 0.000 > ipaddress.py:610(__repr__) > 256000 0.431 0.000 1.554 0.000 > ipaddress.py:613(__str__) > 1792005 0.817 0.000 1.269 0.000 > vpp_serializer.py:111(pack) > 1152001 0.633 0.000 1.167 0.000 > vpp_serializer.py:37(conversion_required) > 256000 0.220 0.000 1.072 0.000 > ipaddress.py:585(__str__) > 256000 0.280 0.000 1.066 0.000 > vpp_serializer.py:521(pack) > 256000 0.487 0.000 1.032 0.000 > vpp_format.py:134(<lambda>) > 256000 0.272 0.000 0.852 0.000 > ipaddress.py:1162(_string_from_ip_int) > 512000 0.400 0.000 0.779 0.000 > vpp_serializer.py:412(pack) > 2304008 0.669 0.000 0.669 0.000 {built-in method > builtins.isinstance} > 256002 0.178 0.000 0.615 0.000 > __init__.py:1286(debug) > > > > Thanks! > -------------------- > Velichkin Viktor, > network engineer > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21489): https://lists.fd.io/g/vpp-dev/message/21489 Mute This Topic: https://lists.fd.io/mt/91461104/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
