> -----Original Message-----
> From: David Marchand <david.march...@redhat.com>
> Sent: Thursday, October 14, 2021 12:45 AM
> To: Peng, ZhihongX <zhihongx.p...@intel.com>
> Cc: Burakov, Anatoly <anatoly.bura...@intel.com>; Ananyev, Konstantin
> <konstantin.anan...@intel.com>; Stephen Hemminger
> <step...@networkplumber.org>; dev <dev@dpdk.org>; Lin, Xueqin
> <xueqin....@intel.com>; Richardson, Bruce <bruce.richard...@intel.com>;
> Thomas Monjalon <tho...@monjalon.net>
> Subject: Re: [PATCH v9 1/3] Enable ASan for memory detector on DPDK
>
> On Tue, Oct 12, 2021 at 11:54 AM <zhihongx.p...@intel.com> wrote:
> >
> > From: Zhihong Peng <zhihongx.p...@intel.com>
> >
> > `AddressSanitizer
> > <https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> > is a widely-used debugging tool to detect memory access errors.
> > It helps detect issues like use-after-free, various kinds of buffer
> > overruns in C/C++ programs, and other similar errors, as well as
> > printing out detailed debug information whenever an error is detected.
> >
>
>
> Comments below concern added feature with patch 2.
> From here...
The v10 version will be fixed.
> > DPDK ASan functionality is currently only supported Linux x86_64.
> > Support other platforms, need to define ASAN_SHADOW_OFFSET value
> > according to google ASan document, and configure meson
> > (config/meson.build).
> >
> > Here is an example of heap-buffer-overflow bug:
> > ......
> > char *p = rte_zmalloc(NULL, 7, 0);
> > p[7] = 'a';
> > ......
> >
> > Here is an example of use-after-free bug:
> > ......
> > char *p = rte_zmalloc(NULL, 7, 0);
> > rte_free(p);
> > *p = 'a';
> > ......
> ... to here.
>
>
> >
> > We can enable ASan by adding below compilation options:
> > -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
> > "-Dbuildtype=debug": This is a non-essential option. When this option
> > is added, if a memory error occurs, ASan can clearly show where the
> > code is wrong.
> > "-Db_lundef=false": When use clang to compile DPDK, this option must
> > be added.
> >
> > Signed-off-by: Xueqin Lin <xueqin....@intel.com>
> > Signed-off-by: Zhihong Peng <zhihongx.p...@intel.com>
>
>
> More problematic, linking an external (out of meson) application to a dpdk
> compiled with ASan is broken.
>
> My environment contains following targets compiled using
> ./devtools/test-meson-builds.sh:
> $ ls $HOME/builds/
> build-arm64-bluefield build-arm64-host-clang build-clang-shared build-gcc-
> shared build-ppc64le-power8 build-x86-mingw
> build-arm64-dpaa build-arm64-octeontx2 build-clang-static
> build-gcc-static build-x86-generic
>
> I stopped at patch 1, configured following target to have ASan in them, like
> this:
>
> $ meson configure $HOME/builds/build-gcc-static -Db_sanitize=address
> $ meson configure $HOME/builds/build-clang-shared -Db_sanitize=address -
> Db_lundef=false $ meson configure $HOME/builds/build-x86-generic -
> Db_sanitize=address
> ^^^^^^^^^^^^^^^^^
> This is the target for which we test linking a
> dpdk application
> out of meson.
>
> $ meson configure $HOME/builds/build-arm64-bluefield -
> Db_sanitize=address
>
> Then ran the check:
> $ ./devtools/test-meson-builds.sh
> ...
> ...
> ...
> ## Building cmdline
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tempnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam_r'
> is dangerous, better use `mkstemp'
> ## Building helloworld
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tempnam' is
> dangerous, better use `mkstemp'
> /usr/bin/ld: /usr/lib64/libasan.so.6: warning: the use of `tmpnam_r'
> is dangerous, better use `mkstemp'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `read_memory_node':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:57:
> undefined reference to `__asan_option_detect_stack_use_after_return'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:65:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:61:
> undefined reference to `__asan_report_store_n'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:60:
> undefined reference to `__asan_report_store_n'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:57:
> undefined reference to `__asan_stack_malloc_3'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:77:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:86:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:86:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:88:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:88:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):/home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:102:
> more undefined references to `__asan_report_load8' follow
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `read_memory_node':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:119:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:145:
> undefined reference to `__asan_report_load16'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:135:
> undefined reference to `__asan_report_store_n'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:143:
> undefined reference to `__asan_report_load8'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `rotate_8':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:40:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:39:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:40:
> undefined reference to `__asan_report_store4'
> /usr/bin/ld: /home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/librte_common_dpaax.a(common_dpaax_dpaax
> _iova_table.c.o):
> in function `read_memory_node':
> /home/dmarchan/builds/build-x86-
> generic/../../dpdk/drivers/common/dpaax/dpaax_iova_table.c:143:
> undefined reference to `__asan_report_load8'
> *flood of errors about undefined symbols from libasan*
>
>
> Tried to debug, in-tree compiled testpmd shows:
> $ readelf -d $HOME/builds/build-x86-generic/app/dpdk-testpmd | grep asan
> 0x0000000000000001 (NEEDED) Shared library: [libasan.so.6]
> But I see nothing in ninja:
> $ grep -i asan $HOME/builds/build-x86-generic/build.ninja
>
> And for external link, looking at an "installed" .pc:
> $ PKG_CONFIG_PATH=/home/dmarchan/builds/build-x86-
> generic/install/usr/local/lib/pkgconfig
> pkg-config libdpdk --libs | grep asan
Need to link -lasan when using gcc, but don’t need to link when using clang.
You added it in the v6 version comments, but need to restrict when using clang.
All code:
if get_option('b_sanitize') == 'address' or get_option('b_sanitize') ==
'address,undefined'
if cc.get_id() == 'gcc'
asan_dep = cc.find_library('asan', required: true)
if (not cc.links('int main(int argc, char *argv[]) { return 0; }',
dependencies: asan_dep))
error('broken dependency, "libasan"')
endif
add_project_link_arguments('-lasan', language: 'c')
dpdk_extra_ldflags += '-lasan'
endif
if exec_env == 'linux' and arch_subdir == 'x86'
dpdk_conf.set10('RTE_MALLOC_ASAN', true)
endif
endif
>
>
> Some more comments about splitting doc with patch2:
>
>
> > ---
> > v7: 1) Split doc and code into two.
> > 2) Modify asan.rst doc
> > v8: No change.
> > v9: 1) Add the check of libasan library.
> > 2) Add release notes.
> > ---
> > config/meson.build | 10 +++
> > devtools/words-case.txt | 1 +
> > doc/guides/prog_guide/asan.rst | 97
> ++++++++++++++++++++++++++
> > doc/guides/prog_guide/index.rst | 1 +
> > doc/guides/rel_notes/release_21_11.rst | 9 +++
> > 5 files changed, 118 insertions(+)
> > create mode 100644 doc/guides/prog_guide/asan.rst
> >
> > diff --git a/config/meson.build b/config/meson.build index
> > 4cdf589e20..5170b79fed 100644
> > --- a/config/meson.build
> > +++ b/config/meson.build
> > @@ -411,6 +411,16 @@ if get_option('b_lto')
> > endif
> > endif
> >
> > +if get_option('b_sanitize') == 'address'
> > + if cc.get_id() == 'gcc'
> > + asan_dep = cc.find_library('asan', required: true)
> > + if (not cc.links('int main(int argc, char *argv[]) { return 0; }',
> > + dependencies: asan_dep))
> > + error('broken dependency, "libasan"')
> > + endif
> > + endif
> > +endif
> > +
> > if get_option('default_library') == 'both'
> > error( '''
> > Unsupported value "both" for "default_library" option.
> > diff --git a/devtools/words-case.txt b/devtools/words-case.txt index
> > 0bbad48626..ada6910fa0 100644
> > --- a/devtools/words-case.txt
> > +++ b/devtools/words-case.txt
> > @@ -5,6 +5,7 @@ API
> > Arm
> > armv7
> > armv8
> > +ASan
> > BAR
> > CRC
> > DCB
> > diff --git a/doc/guides/prog_guide/asan.rst
> > b/doc/guides/prog_guide/asan.rst new file mode 100644 index
> > 0000000000..627675ef46
> > --- /dev/null
> > +++ b/doc/guides/prog_guide/asan.rst
> > @@ -0,0 +1,97 @@
> > +.. Copyright (c) <2021>, Intel Corporation
> > + All rights reserved.
> > +
> > +Memory error detect standard tool - AddressSanitizer(ASan)
> >
> +=========================================================
> =
> > +
> > +`AddressSanitizer
> > +<https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> > +is a widely-used debugging tool to detect memory access errors.
> > +It helps detect issues like use-after-free, various kinds of buffer
> > +overruns in C/C++ programs, and other similar errors, as well as
> > +printing out detailed debug information whenever an error is detected.
> > +
> > +AddressSanitizer is a part of LLVM (3.1+) and GCC (4.8+).
>
> Same problem, below should be in patch 2, from here...
The v10 version will be fixed.
>
> > +
> > +DPDK ASan functionality is currently only supported Linux x86_64.
> > +Support other platforms, need to define ASAN_SHADOW_OFFSET value
> > +according to google ASan document, and configure meson
> > +(config/meson.build).
> > +
> > +Example heap-buffer-overflow error
> > +----------------------------------
> > +
> > +Following error was reported when ASan was enabled::
> > +
> > + Applied 9 bytes of memory, but accessed the 10th byte of memory,
> > + so heap-buffer-overflow appeared.
> > +
> > +Below code results in this error::
> > +
> > + Add code to helloworld:
> > + char *p = rte_zmalloc(NULL, 9, 0);
> > + if (!p) {
> > + printf("rte_zmalloc error.");
> > + return -1;
> > + }
> > + p[9] = 'a';
> > +
> > +The error log::
> > +
> > + ==369953==ERROR: AddressSanitizer: heap-buffer-overflow on address
> 0x7fb17f465809 at pc 0x5652e6707b84 bp 0x7ffea70eea20 sp 0x7ffea70eea10
> WRITE of size 1 at 0x7fb17f465809 thread T0
> > + #0 0x5652e6707b83 in main ../examples/helloworld/main.c:47
> > + #1 0x7fb94953c0b2 in __libc_start_main (/lib/x86_64-linux-
> gnu/libc.so.6+0x270b2)
> > + #2 0x5652e67079bd in _start
> > + (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-
> hellow
> > + orld+0x8329bd)
> > +
> > + Address 0x7fb17f465809 is a wild pointer.
> > + SUMMARY: AddressSanitizer: heap-buffer-overflow
> > + ../examples/helloworld/main.c:47 in main
> > +
> > +Example use-after-free error
> > +----------------------------
> > +
> > +Following error was reported when ASan was enabled::
> > +
> > + Applied for 9 bytes of memory, and accessed the first byte after
> > + released, so heap-use-after-free appeared.
> > +
> > +Below code results in this error::
> > +
> > + Add code to helloworld:
> > + char *p = rte_zmalloc(NULL, 9, 0);
> > + if (!p) {
> > + printf("rte_zmalloc error.");
> > + return -1;
> > + }
> > + rte_free(p);
> > + *p = 'a';
> > +
> > +The error log::
> > +
> > + ==417048==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x7fc83f465800 at pc 0x564308a39b89 bp 0x7ffc8c85bf50 sp 0x7ffc8c85bf40
> WRITE of size 1 at 0x7fc83f465800 thread T0
> > + #0 0x564308a39b88 in main ../examples/helloworld/main.c:48
> > + #1 0x7fd0079c60b2 in __libc_start_main (/lib/x86_64-linux-
> gnu/libc.so.6+0x270b2)
> > + #2 0x564308a399bd in _start
> > + (/home/pzh/asan_test/x86_64-native-linuxapp-gcc/examples/dpdk-
> hellow
> > + orld+0x8329bd)
> > +
> > + Address 0x7fc83f465800 is a wild pointer.
> > + SUMMARY: AddressSanitizer: heap-use-after-free
> > + ../examples/helloworld/main.c:48 in main
>
> ... till here.
>
>
> > +
> > +Usage
> > +-----
> > +
> > +meson build
> > +^^^^^^^^^^^
> > +
> > +To enable ASan in meson build system, use following meson build
> command:
> > +
> > +Example usage::
> > +
> > + meson build -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
>
> l_undef unnecessary for gcc.
The v10 version will be fixed.
>
> > + ninja -C build
> > +
> > +.. Note::
> > +
> > + a) Some of the features of ASan (for example, 'Display memory
> application location, currently
> > + displayed as a wild pointer') are not currently supported by DPDK's
> implementation.
>
> This comment should be in patch 2, there is nothing specific in patch
> 1 about DPDK wrt ASan.
The v10 version will be fixed.
>
> > + b) DPDK test has been completed in
> ubuntu18.04/ubuntu20.04/redhat8.3. To compile with gcc in
> > + centos, libasan needs to be installed separately.
> > + c) If the program uses cmdline, when a memory bug occurs, need to
> execute the "stty echo" command.
> > diff --git a/doc/guides/prog_guide/index.rst
> > b/doc/guides/prog_guide/index.rst index 2dce507f46..df8a4b93e1 100644
> > --- a/doc/guides/prog_guide/index.rst
> > +++ b/doc/guides/prog_guide/index.rst
> > @@ -71,3 +71,4 @@ Programmer's Guide
> > lto
> > profile_app
> > glossary
> > + asan
> > diff --git a/doc/guides/rel_notes/release_21_11.rst
> > b/doc/guides/rel_notes/release_21_11.rst
> > index 5036641842..58c6377148 100644
> > --- a/doc/guides/rel_notes/release_21_11.rst
> > +++ b/doc/guides/rel_notes/release_21_11.rst
> > @@ -141,6 +141,15 @@ New Features
> > * Added tests to validate packets hard expiry.
> > * Added tests to verify tunnel header verification in IPsec inbound.
> >
> > +* **Enable ASan for memory detector on DPDK.**
> > +
> > + `AddressSanitizer
> > + <https://github.com/google/sanitizers/wiki/AddressSanitizer>`
> > + (ASan) is a widely-used debugging tool to detect memory access errors.
> > + It helps detect issues like use-after-free, various kinds of buffer
> > + overruns in C/C++ programs, and other similar errors, as well as
> > + printing out detailed debug information whenever an error is detected.
> > +
> >
> > Removed Items
> > -------------
> > --
> > 2.25.1
> >
>
>
> --
> David Marchand
