Hi Ben,
I tried to run ASAN on vpp version 21.10 in debug mode.But It's not working
for me.
Issue 1:
I compiled vpp code with below build command (with clang)
*sudo make rebuild VPP_EXTRA_CMAKE_ARGS=-DVPP_ENABLE_SANITIZE_ADDR=ON*
After compilation starting vpp is failing with below error
*sed: symbol lookup error:
/home/supervisor/libvpp/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.1.0.1:
undefined symbol: __asan_option_detect_stack_use_after_return*
Issue 2:
since compiling with clang is not working i tried compiling with GCC. i
used below command for compilation
*sudo make rebuild VPP_EXTRA_CMAKE_ARGS=-DVPP_ENABLE_SANITIZE_ADDR=ON
CC=gcc-8*
with this vpp startup is failing with below error
*==908==ASan runtime does not come first in initial library list; you
should either link runtime to your application or manually preload it
with LD_PRELOAD.*
I resolved above issue with below configuration
LD_PRELOAD=/usr/lib/gcc/x86_64-linux-gnu/8/libasan.so
export LD_PRELOAD
Issue 3:
once issue 2 is resolved vpp crashed with below backtrace
```
Thread 1 (Thread 0x7ffff7fce400 (LWP 678)):
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff75917f1 in __GI_abort () at abort.c:79
#2 0x00007fffe99f39ce in ?? () from /usr/lib/x86_64-linux-gnu/libasan.so.5
#3 0x00007fffe99fc088 in ?? () from /usr/lib/x86_64-linux-gnu/libasan.so.5
#4 0x00007fffe99d3a39 in ?? () from /usr/lib/x86_64-linux-gnu/libasan.so.5
#5 0x00007fffe99e1ee9 in ?? () from /usr/lib/x86_64-linux-gnu/libasan.so.5
#6 0x00007ffff7de38d3 in call_init (env=0x7fffffffe308,
argv=0x7fffffffe2e8, argc=3, l=<optimized out>) at dl-init.c:72
#7 _dl_init (main_map=main_map@entry=0x5555583cf670, argc=3,
argv=0x7fffffffe2e8, env=0x7fffffffe308) at dl-init.c:119
#8 0x00007ffff7de839f in dl_open_worker (a=a@entry=0x7fffffffcf40) at
dl-open.c:522
#9 0x00007ffff76b816f in __GI__dl_catch_exception
(exception=0x7fffffffcf20, operate=0x7ffff7de7f60 <dl_open_worker>,
args=0x7fffffffcf40)
at dl-error-skeleton.c:196
#10 0x00007ffff7de796a in _dl_open (file=0x7fffffffd1b0
"/usr/local/lib/librtbvpp.so", mode=-2147483391,
caller_dlopen=0x7ffff794b73d <cmd_open_plugin+86>, nsid=<optimized
out>, argc=3, argv=<optimized out>, env=0x7fffffffe308) at
dl-open.c:605
#11 0x00007ffff70c9f96 in dlopen_doit (a=a@entry=0x7fffffffd170) at dlopen.c:66
#12 0x00007ffff76b816f in __GI__dl_catch_exception
(exception=exception@entry=0x7fffffffd110, operate=0x7ffff70c9f40
<dlopen_doit>,
args=0x7fffffffd170) at dl-error-skeleton.c:196
#13 0x00007ffff76b81ff in __GI__dl_catch_error
(objname=0x55555779df30, errstring=0x55555779df38,
mallocedp=0x55555779df28, operate=<optimized out>,
args=<optimized out>) at dl-error-skeleton.c:215
#14 0x00007ffff70ca745 in _dlerror_run
(operate=operate@entry=0x7ffff70c9f40 <dlopen_doit>,
args=args@entry=0x7fffffffd170) at dlerror.c:162
#15 0x00007ffff70ca051 in __dlopen (file=file@entry=0x7fffffffd1b0
"/usr/local/lib/librtbvpp.so", mode=mode@entry=257) at dlopen.c:87
```
I resolved this issue by exporting below ASAN options
*export ASAN_OPTIONS=verify_asan_link_order=0:detect_odr_violation=0*
Once issue 3 is resolved vpp is up but it's not catching/reporting any
memory leaks
(I induced one leak in our code to verify the same).
we are starting our fib process (vpp) by executing below script
supervisor@dev1_sudhir>srv2:~ $ cat fib.sh
LD_PRELOAD=/usr/lib/gcc/x86_64-linux-gnu/8/libasan.so
export LD_PRELOAD
export ASAN_OPTIONS=verify_asan_link_order=0:detect_odr_violation=0
sudo -E gdb --args bd -i /etc/rtbrick/bd/config/fibd.json
./fib.sh
Can you please let me know if you find any issue in the procedure i followed or
any pointers to solve the issue i am facing.
Thanks and Regards,
Sudhir
On Thu, Feb 24, 2022 at 6:39 PM Sudhir CR via lists.fd.io <sudhir=
rtbrick....@lists.fd.io> wrote:
> Hi Ben,
> Thanks for the update.
>
> Regards,
> Sudhir
>
> On Thu, Feb 24, 2022 at 6:37 PM Benoit Ganne (bganne) <bga...@cisco.com>
> wrote:
>
>> Hi Sudhir,
>>
>> I am working on a few bugfixes for ASan.
>> Right now, I'd recommend to use ASan in debug builds only, there are
>> several issues that need to be fixed in release mode.
>>
>> Best
>> ben
>>
>> > -----Original Message-----
>> > From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Sudhir CR
>> via
>> > lists.fd.io
>> > Sent: jeudi 24 février 2022 14:03
>> > To: vpp-dev@lists.fd.io
>> > Subject: [vpp-dev] VPP 2110 with AddressSanitizer enabled
>> >
>> > Hi Team,
>> >
>> >
>> > We compiled vpp code with AddressSanitizer enabled. We used the below
>> > command to compile code.
>> >
>> >
>> > sudo make rebuild-release VPP_EXTRA_CMAKE_ARGS=-
>> > DVPP_ENABLE_SANITIZE_ADDR=ON
>> >
>> >
>> >
>> > we are starting vpp with below command
>> >
>> >
>> > sudo ./vpp -c /etc/vpp/startup.conf
>> >
>> >
>> >
>> > But vpp startup is failed with below AddressSanitizer error
>> >
>> >
>> > AddressSanitizer:DEADLYSIGNAL
>> > =================================================================
>> > ==1442028==ERROR: AddressSanitizer: SEGV on unknown address
>> 0x0200255e3b3e
>> > (pc 0x7fe59ae338f7 bp 0x7ffc128b1c40 sp 0x7ffc128b1480 T0)
>> > ==1442028==The signal is caused by a READ memory access.
>> > #0 0x7fe59ae338f6 in sanitizer_unpoison_push__
>> > /home/supervisor/development/libvpp/src/vppinfra/sanitizer.h:54:17
>> > #1 0x7fe59ae338f6 in hash_memory64
>> > /home/supervisor/development/libvpp/src/vppinfra/hash.c:157:15
>> > #2 0x7fe59ae338f6 in hash_memory
>> > /home/supervisor/development/libvpp/src/vppinfra/hash.c:280:10
>> > #3 0x7fe59ae34b4e in key_sum
>> > /home/supervisor/development/libvpp/src/vppinfra/hash.c
>> > #4 0x7fe59ae34b4e in lookup.llvm.8926505759877686271
>> > /home/supervisor/development/libvpp/src/vppinfra/hash.c:557:7
>> > #5 0x7fe59aeb203f in _hash_set3
>> > /home/supervisor/development/libvpp/src/vppinfra/hash.c:848:10
>> > #6 0x7fe59c4edc8e in config_one_plugin
>> > /home/supervisor/development/libvpp/src/vlib/unix/plugin.c:710:3
>> > #7 0x7fe59c4edc8e in vlib_plugin_config
>> > /home/supervisor/development/libvpp/src/vlib/unix/plugin.c:775:12
>> > #8 0x7fe59c49c3f6 in vlib_unix_main
>> > /home/supervisor/development/libvpp/src/vlib/unix/main.c:764:12
>> > #9 0x55a040ce2e40 in main
>> > /home/supervisor/development/libvpp/src/vpp/vnet/main.c:344:14
>> > #10 0x7fe59a29ebf6 in __libc_start_main /build/glibc-S9d2JN/glibc-
>> > 2.27/csu/../csu/libc-start.c:310
>> > #11 0x55a040c41479 in _start
>> > (/home/supervisor/development/libvpp/build-root/install-vpp-
>> > native/vpp/bin/vpp+0x41479)
>> >
>> > AddressSanitizer can not provide additional info.
>> > SUMMARY: AddressSanitizer: SEGV
>> > /home/supervisor/development/libvpp/src/vppinfra/sanitizer.h:54:17 in
>> > sanitizer_unpoison_push__
>> > ==1442028==ABORTING
>> > Aborted
>> >
>> >
>> >
>> > contents of startup.conf looks like below
>> >
>> >
>> >
>> > unix {
>> > nodaemon
>> > log /var/log/vpp/vpp.log
>> > full-coredump
>> > cli-listen localhost:5003
>> > runtime-dir /shm/run/vpp/
>> > }
>> >
>> > api-trace {
>> > on
>> > }
>> >
>> > cpu {
>> > main-core 2
>> > corelist-workers 3,4
>> > }
>> >
>> > heapsize 600M
>> >
>> > statseg {
>> > size 150M
>> > }
>> > plugin_path /usr/local/lib/vpp_plugins/
>> > plugins {
>> > plugin rtbrick_bcm_plugin.so { disable }
>> > plugin dpdk_plugin.so { disable }
>> > }
>> >
>> >
>> >
>> > Please let me know any suggestions on how to resolve this Error.
>> >
>> >
>> > Thanks in Advance,
>> >
>> > Sudhir
>> >
>> >
>> > NOTICE TO RECIPIENT This e-mail message and any attachments are
>> > confidential and may be privileged. If you received this e-mail in
>> error,
>> > any review, use, dissemination, distribution, or copying of this e-mail
>> is
>> > strictly prohibited. Please notify us immediately of the error by return
>> > e-mail and please delete this message from your system. For more
>> > information about Rtbrick, please visit us at www.rtbrick.com
>> > <http://www.rtbrick.com>
>>
>
> NOTICE TO RECIPIENT This e-mail message and any attachments are
> confidential and may be privileged. If you received this e-mail in error,
> any review, use, dissemination, distribution, or copying of this e-mail is
> strictly prohibited. Please notify us immediately of the error by return
> e-mail and please delete this message from your system. For more
> information about Rtbrick, please visit us at www.rtbrick.com
>
>
>
>
--
NOTICE TO
RECIPIENT This e-mail message and any attachments are
confidential and may be
privileged. If you received this e-mail in error,
any review, use,
dissemination, distribution, or copying of this e-mail is
strictly
prohibited. Please notify us immediately of the error by return
e-mail and
please delete this message from your system. For more
information about Rtbrick, please visit us at www.rtbrick.com
<http://www.rtbrick.com>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21017): https://lists.fd.io/g/vpp-dev/message/21017
Mute This Topic: https://lists.fd.io/mt/89364107/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
