Hi!
Can anyone please reveiw this patch?
Explanation:
For small amount of objects (for example short ACL) there is no problems in python api, but if you trying to load thousands - papi works very slow. Loading ACL with 256k entries could take forever.
Here below is comparsion of uploading 128k entries ACL:
- excuted command:
r = vpp.api.acl_add_replace(acl_index=0xffffffff, count=len(rules[:128000]) , r=rules[:128000])
- without fix it took 121 seconds, and scales quadratically:
15488160 function calls (14592160 primitive calls) in 121.454 secondsOrdered by: cumulative timencalls tottime percall cumtime percall filename:lineno(function)1 0.000 0.000 121.454 121.454 {built-in method builtins.exec}1 0.022 0.022 121.454 121.454 <string>:1(<module>)1 0.000 0.000 121.432 121.432 vpp_papi.py:120(__call__)1 0.000 0.000 121.432 121.432 vpp_papi.py:494(f)1 0.001 0.001 121.432 121.432 vpp_papi.py:694(_call_vpp)896001/1 6.084 0.000 118.642 118.642 vpp_serializer.py:627(pack)1 103.522 103.522 118.640 118.640 vpp_serializer.py:298(pack)256000 0.469 0.000 9.855 0.000 vpp_serializer.py:47(conversion_packer)256000 1.368 0.000 3.942 0.000 vpp_serializer.py:472(pack)256008 1.338 0.000 3.349 0.000 {method 'format' of 'str' objects}1 0.000 0.000 2.715 2.715 vpp_papi.py:728(<listcomp>)256000 0.448 0.000 2.011 0.000 ipaddress.py:610(__repr__)256000 0.435 0.000 1.563 0.000 ipaddress.py:613(__str__)1792005 0.819 0.000 1.330 0.000 vpp_serializer.py:111(pack)1152001 0.690 0.000 1.281 0.000 vpp_serializer.py:37(conversion_required)256000 0.610 0.000 1.249 0.000 vpp_format.py:134(<lambda>)256000 0.323 0.000 1.151 0.000 vpp_serializer.py:521(pack)256000 0.215 0.000 1.077 0.000 ipaddress.py:585(__str__)512000 0.475 0.000 0.897 0.000 vpp_serializer.py:412(pack)256000 0.281 0.000 0.862 0.000 ipaddress.py:1162(_string_from_ip_int)2304008 0.774 0.000 0.774 0.000 {built-in method builtins.isinstance}256002 0.194 0.000 0.697 0.000 __init__.py:1286(debug)
- with proposed fix it took just 16.5 seconds, and what more important it scales linearly (that means 256k will upload for ~ 32 seconds)
15488160 function calls (14592160 primitive calls) in 16.516 secondsOrdered by: cumulative timencalls tottime percall cumtime percall filename:lineno(function)1 0.000 0.000 16.516 16.516 {built-in method builtins.exec}1 0.012 0.012 16.516 16.516 <string>:1(<module>)1 0.000 0.000 16.504 16.504 vpp_papi.py:120(__call__)1 0.000 0.000 16.504 16.504 vpp_papi.py:494(f)1 0.001 0.001 16.504 16.504 vpp_papi.py:694(_call_vpp)896001/1 5.690 0.000 13.723 13.723 vpp_serializer.py:627(pack)1 0.133 0.133 13.719 13.719 vpp_serializer.py:298(pack)256000 0.474 0.000 9.162 0.000 vpp_serializer.py:47(conversion_packer)256000 1.155 0.000 3.370 0.000 vpp_serializer.py:472(pack)256008 1.162 0.000 3.161 0.000 {method 'format' of 'str' objects}1 0.000 0.000 2.706 2.706 vpp_papi.py:728(<listcomp>)256000 0.445 0.000 1.999 0.000 ipaddress.py:610(__repr__)256000 0.431 0.000 1.554 0.000 ipaddress.py:613(__str__)1792005 0.817 0.000 1.269 0.000 vpp_serializer.py:111(pack)1152001 0.633 0.000 1.167 0.000 vpp_serializer.py:37(conversion_required)256000 0.220 0.000 1.072 0.000 ipaddress.py:585(__str__)256000 0.280 0.000 1.066 0.000 vpp_serializer.py:521(pack)256000 0.487 0.000 1.032 0.000 vpp_format.py:134(<lambda>)256000 0.272 0.000 0.852 0.000 ipaddress.py:1162(_string_from_ip_int)512000 0.400 0.000 0.779 0.000 vpp_serializer.py:412(pack)2304008 0.669 0.000 0.669 0.000 {built-in method builtins.isinstance}256002 0.178 0.000 0.615 0.000 __init__.py:1286(debug)
Thanks!
--------------------
Velichkin Viktor,
network engineer
Velichkin Viktor,
network engineer
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21487): https://lists.fd.io/g/vpp-dev/message/21487 Mute This Topic: https://lists.fd.io/mt/91461104/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
