Hi David, > -----Original Message----- > From: David Marchand <david.march...@redhat.com> > Sent: Tuesday, September 13, 2022 10:56 PM > To: Rong, Leyi <leyi.r...@intel.com> > Cc: ferruh.yi...@xilinx.com; suanmi...@nvidia.com; Wang, Yipeng1 > <yipeng1.w...@intel.com>; zaoxing...@gmail.com; Gobriel, Sameh > <sameh.gobr...@intel.com>; dev@dpdk.org; Richardson, Bruce > <bruce.richard...@intel.com>; Aaron Conole <acon...@redhat.com>; Michael > Santana <maicolgabr...@hotmail.com>; Lincoln Lavoie <lylav...@iol.unh.edu> > Subject: Re: [PATCH v2 1/2] member: implement NitroSketch mode > > On Wed, Aug 31, 2022 at 8:47 AM Leyi Rong <leyi.r...@intel.com> wrote: > > > > Sketching algorithm provide high-fidelity approximate measurements and > > appears as a promising alternative to traditional approaches such as > > packet sampling. > > > > NitroSketch [1] is a software sketching framework that optimizes > > performance, provides accuracy guarantees, and supports a variety of > > sketches. > > > > This commit adds a new data structure called sketch into membership > > library. This new data structure is an efficient way to profile the > > traffic for heavy hitters. Also use min-heap structure to maintain the > > top-k flow keys. > > http://mails.dpdk.org/archives/test-report/2022-August/304026.html > > This patch adds new symbols in the API without going through the experimental > phase. > What is the rationale for skipping it? > Yes, these new APIs should be marked as "experimental", will fix it.
> > > > [1] Zaoxing Liu, Ran Ben-Basat, Gil Einziger, Yaron Kassner, Vladimir > > Braverman, Roy Friedman, Vyas Sekar, "NitroSketch: Robust and General > > Sketch-based Monitoring in Software Switches", in ACM SIGCOMM 2019. > > https://dl.acm.org/doi/pdf/10.1145/3341302.3342076 > > > > Signed-off-by: Alan Liu <zaoxing...@gmail.com> > > Signed-off-by: Yipeng Wang <yipeng1.w...@intel.com> > > Signed-off-by: Leyi Rong <leyi.r...@intel.com> > > --- > > lib/member/meson.build | 38 +- > > lib/member/rte_member.c | 75 ++++ > > lib/member/rte_member.h | 151 ++++++- > > lib/member/rte_member_heap.h | 424 ++++++++++++++++++ > > lib/member/rte_member_sketch.c | 594 ++++++++++++++++++++++++++ > > lib/member/rte_member_sketch.h | 97 +++++ > > lib/member/rte_member_sketch_avx512.c | 69 +++ > > lib/member/rte_member_sketch_avx512.h | 36 ++ > > lib/member/rte_xxh64_avx512.h | 117 +++++ > > lib/member/version.map | 3 + > > 10 files changed, 1600 insertions(+), 4 deletions(-) create mode > > 100644 lib/member/rte_member_heap.h create mode 100644 > > lib/member/rte_member_sketch.c create mode 100644 > > lib/member/rte_member_sketch.h create mode 100644 > > lib/member/rte_member_sketch_avx512.c > > create mode 100644 lib/member/rte_member_sketch_avx512.h > > create mode 100644 lib/member/rte_xxh64_avx512.h > > > > diff --git a/lib/member/meson.build b/lib/member/meson.build index > > e06fddc240..9b3418c25c 100644 > > --- a/lib/member/meson.build > > +++ b/lib/member/meson.build > > @@ -7,6 +7,42 @@ if is_windows > > subdir_done() > > endif > > > > -sources = files('rte_member.c', 'rte_member_ht.c', > > 'rte_member_vbf.c') > > +sources = files('rte_member.c', 'rte_member_ht.c', > > +'rte_member_vbf.c', 'rte_member_sketch.c') > > headers = files('rte_member.h') > > deps += ['hash'] > > +includes += include_directories('../hash', '../ring') > > + > > +# compile AVX512 version if: > > +# we are building 64-bit binary AND binutils can generate proper code > > +if dpdk_conf.has('RTE_ARCH_X86_64') and binutils_ok > > + # compile AVX512 version if either: > > + # a. we have AVX512 supported in minimum instruction set > > + # baseline > > + # b. it's not minimum instruction set, but supported by > > + # compiler > > + # > > + # in former case, just add avx512 C file to files list > > + # in latter case, compile c file to static lib, using correct > > + # compiler flags, and then have the .o file from static lib > > + # linked into main lib. > > + sketch_avx512_cpu_support = ( > > + cc.get_define('__AVX512F__', args: machine_args) != '' > > + ) > > + > > + if sketch_avx512_cpu_support == true > > + cflags += ['-DCC_AVX512_SUPPORT'] > > + if cc.has_multi_arguments('-mavx512f', '-mavx512dq', '-mavx512ifma') > > + cflags += ['-mavx512f', '-mavx512dq', '-mavx512ifma'] > > Pushing those flags in the cflags is probably wrong, as the rest of the > library > objects will be compiled with those AVX512 flags. > If later this library code is run on a non supporting AVX512 system, it will > trigger > a runtime error. > Yes, thanks for the reminder, will fix it. > > Please look at how other libraries integrated AVX512. > Thanks. > > -- > David Marchand
