On Mon, Mar 8, 2021 at 5:38 PM Luca Boccassi <bl...@debian.org> wrote:
>
> On Mon, 2021-03-08 at 03:23 +0000, Ruifeng Wang wrote:
> > > -----Original Message-----
> > > From: Jerin Jacob Kollanukkaran <jer...@marvell.com>
> > > Sent: Sunday, March 7, 2021 9:35 PM
> > > To: Ruifeng Wang <ruifeng.w...@arm.com>; Juraj Linkeš
> > > <juraj.lin...@pantheon.tech>; Luca Boccassi <bl...@debian.org>;
> > > sta...@dpdk.org; dev@dpdk.org; tho...@monjalon.net; Ashwin Sekhar
> > > Thalakalath Kottilveetil <asek...@marvell.com>; Andrew Pinski
> > > <apin...@marvell.com>
> > > Cc: david.march...@redhat.com; nd <n...@arm.com>; nd <n...@arm.com>
> > > Subject: RE: [PATCH 20.11] config/arm: replace native machine args
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Ruifeng Wang <ruifeng.w...@arm.com>
> > > > Sent: Monday, March 1, 2021 11:10 AM
> > > > To: Jerin Jacob Kollanukkaran <jer...@marvell.com>; Juraj Linkeš
> > > > <juraj.lin...@pantheon.tech>; Luca Boccassi <bl...@debian.org>;
> > > > sta...@dpdk.org
> > > > Cc: david.march...@redhat.com; nd <n...@arm.com>; nd <n...@arm.com>
> > > > Subject: [EXT] RE: [PATCH 20.11] config/arm: replace native machine
> > > > args
> > > >
> > > > External Email
> > > >
> > > > ----------------------------------------------------------------------
> > > > > -----Original Message-----
> > > > > From: Jerin Jacob Kollanukkaran <jer...@marvell.com>
> > > > > Sent: Thursday, February 25, 2021 8:15 PM
> > > > > To: Ruifeng Wang <ruifeng.w...@arm.com>; Juraj Linkeš
> > > > > <juraj.lin...@pantheon.tech>; Luca Boccassi <bl...@debian.org>;
> > > > > sta...@dpdk.org
> > > > > Cc: david.march...@redhat.com; nd <n...@arm.com>
> > > > > Subject: RE: [PATCH 20.11] config/arm: replace native machine args
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Ruifeng Wang <ruifeng.w...@arm.com>
> > > > > > Sent: Saturday, February 20, 2021 9:13 AM
> > > > > > To: Juraj Linkeš <juraj.lin...@pantheon.tech>; Luca Boccassi
> > > > > > <bl...@debian.org>; sta...@dpdk.org; Jerin Jacob Kollanukkaran
> > > > > > <jer...@marvell.com>
> > > > > > Cc: david.march...@redhat.com; nd <n...@arm.com>
> > > > > > Subject: [EXT] RE: [PATCH 20.11] config/arm: replace native
> > > > > > machine args
> > > > > >
> > > > > > External Email
> > > > > >
> > > > > > ------------------------------------------------------------------
> > > > > > --
> > > > > > --
> > > > > > > -----Original Message-----
> > > > > > > From: Juraj Linkeš <juraj.lin...@pantheon.tech>
> > > > > > > Sent: Friday, February 19, 2021 8:10 PM
> > > > > > > To: Luca Boccassi <bl...@debian.org>; sta...@dpdk.org
> > > > > > > Cc: jer...@marvell.com; Ruifeng Wang <ruifeng.w...@arm.com>;
> > > > > > > david.march...@redhat.com
> > > > > > > Subject: RE: [PATCH 20.11] config/arm: replace native machine
> > > > > > > args
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Luca Boccassi <bl...@debian.org>
> > > > > > > > Sent: Friday, February 19, 2021 12:33 PM
> > > > > > > > To: Juraj Linkeš <juraj.lin...@pantheon.tech>; sta...@dpdk.org
> > > > > > > > Cc: jer...@marvell.com; ruifeng.w...@arm.com;
> > > > > > > > david.march...@redhat.com
> > > > > > > > Subject: Re: [PATCH 20.11] config/arm: replace native machine
> > > > > > > > args
> > > > > > > >
> > > > > > > > On Fri, 2021-02-19 at 11:06 +0000, Juraj Linkeš wrote:
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: luca.bocca...@gmail.com <luca.bocca...@gmail.com>
> > > > > > > > > > Sent: Friday, February 19, 2021 11:58 AM
> > > > > > > > > > To: sta...@dpdk.org
> > > > > > > > > > Cc: Juraj Linkeš <juraj.lin...@pantheon.tech>;
> > > > > > > > > > jer...@marvell.com; ruifeng.w...@arm.com;
> > > > > > > > > > david.march...@redhat.com
> > > > > > > > > > Subject: [PATCH 20.11] config/arm: replace native machine
> > > > > > > > > > args
> > > > > > > > > >
> > > > > > > > > > From: Juraj Linkeš <juraj.lin...@pantheon.tech>
> > > > > > > > > >
> > > > > > > > > > [ backported from upstream commit
> > > > > > > > > > 9186e5a07f35ae74a1f7fa2d89671b5f77eae407 ]
> > > > > > > > > >
> > > > > > > > > > There are compiler issues when building with -mcpu=native
> > > > > > > > > > with popular compilers, such as GCC-8.4:
> > > > > > > > > > In file included from
> > > > > > > > > > ../lib/librte_eal/arm/include/rte_vect.h:11,
> > > > > > > > > > from ../lib/librte_net/net_crc_neon.c:10:
> > > > > > > > > > ../lib/librte_net/net_crc_neon.c: In function
> > > ‘crcr32_folding_round’:
> > > > > > > > > > /usr/lib/gcc/aarch64-linux-gnu/8/include/arm_neon.h:26094:1:
> > > > > error:
> > > > > > > > > > inlining failed in call to always_inline ‘vmull_p64’:
> > > > > > > > > > target specific option mismatch
> > > > > > > > > > vmull_p64 (poly64_t a, poly64_t b)
> > > > > > > > > > ../lib/librte_net/net_crc_neon.c:50:20: note: called from
> > > > > > > > > > here
> > > > > > > > > > uint64x2_t tmp1 = vreinterpretq_u64_p128(vmull_p64(
> > > > > > > > > > vgetq_lane_p64(vreinterpretq_p64_u64(fold), 0),
> > > > > > > > > > vgetq_lane_p64(vreinterpretq_p64_u64(precomp), 1)));
> > > > > > > > > >
> > > > > > > > > > and clang:
> > > > > > > > > > gcc -E -dM -mcpu="native" - < /dev/null | grep
> > > > > > > > > > __ARM_FEATURE_ATOMICS
> > > > > > > > > > clang-9 -E -dM -mcpu="native" - < /dev/null | grep
> > > > > > > > > > __ARM_FEATURE_ATOMICS <no output> # no clang support
> > > > > > > > > >
> > > > > > > > > > Fix this by always specifying the proper machine args and
> > > > > > > > > > never using the native flags.
> > > > > > > > > >
> > > > > > > > > > Fixes: 78ac8eac7e8a ("config/arm: use native machine build
> > > > > > > > > > arguments")
> > > > > > > > > >
> > > > > > > > > > Signed-off-by: Juraj Linkeš <juraj.lin...@pantheon.tech>
> > > > > > > > > > Signed-off-by: Luca Boccassi <luca.bocca...@microsoft.com>
> > > > > > > > > > ---
> > > > > > > > > > This is a crude backport, but it fixes the build for arm64.
> > > > > > > > > > It's a release blocker for 20.11.1, so I would appreciate
> > > > > > > > > > a quick
> > > > > review.
> > > > > > > > > > Thanks!
> > > > > > > > >
> > > > > > > > > What does this fix? With or without the below change, the
> > > > > > > > > native machine
> > > > > > > > args are not used. The patch shoudn't actually change the
> > > > > > > > configuration of the build at all, so I'm a bit confused.
> > > > > > > >
> > > > > > > > It fixes the build on some build workers with thunderx
> > > > > > > > hardware
> > > > > > > > - without this I get failures like:
> > > > > > > >
> > > > > > > > arm_neon.h:26647:1: error: inlining failed in call to
> > > > > > > > 'always_inline'
> > > > > > > > 'vmull_p64': target specific option mismatch
> > > > > > > >
> > > > > > >
> > > > > > > I tried the patch and I'm seeing the same errors on a ThunderX
> > > > > > > server (with and without the patch). Is this actually the right
> > > > > > > patch?
> > > > > > >
> > > > > > > One of the four failures looks like this:
> > > > > > > In file included from ../lib/librte_eal/arm/include/rte_vect.h:11,
> > > > > > > from ../lib/librte_net/net_crc_neon.c:10:
> > > > > > > ../lib/librte_net/net_crc_neon.c: In function
> > > > > > > 'crcr32_folding_round':
> > > > > > > /usr/lib/gcc/aarch64-linux-gnu/8/include/arm_neon.h:26094:1:
> > > > > > > error:
> > > > > > > inlining failed in call to always_inline 'vmull_p64': target
> > > > > > > specific option mismatch
> > > > > > > vmull_p64 (poly64_t a, poly64_t b) ^~~~~~~~~
> > > > > > > ../lib/librte_net/net_crc_neon.c:50:20: note: called from here
> > > > > > > uint64x2_t tmp1 = vreinterpretq_u64_p128(vmull_p64(
> > > > > > > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > vgetq_lane_p64(vreinterpretq_p64_u64(fold), 0),
> > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > vgetq_lane_p64(vreinterpretq_p64_u64(precomp), 1)));
> > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > >
> > > > > > > Ruifeng, any ideas on how to fix this?
> > > > > >
> > > > > > Gcc build on ThunderX platform is broken. Issue can be seen in
> > > > > > some
> > > > > > CentOS-8 OBS builds.
> > > > > > https://urldefense.proofpoint.com/v2/url?u=https-
> > > > > > 3A__mails.dpdk.org_archives_dev_2020-
> > > > > >
> > > 2DNovember_192909.html&d=DwIGaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=1
> > > > > DG
> > > ob4H4rxz6H8uITozGOCa0s5f4wCNtTa4UUKvcsvI&m=mgzJ6z43dsDFwI6rdgKC
> > > > > Uj
> > > > > > 0GCMNjEKQAa7dfRZxvrdU&s=UWUJTFdGC2mD2x-rcuRnH1I7-
> > > > > > 1jKFC40Bh5hFanzu0A&e=
> > > > > > I tried tuning compiler flags used, but could not resolve the issue.
> > > > > >
> > > > > > Need help from Marvell to look at this.
> > > > > > Hi Jerin, do you have any thoughts on this?
> > > > >
> > > > > Ruifeng, If you are able to reproduce this issue, Could you add "-
> > > > > march=armv8.1-a+crc+crypto" In ThunderX config and check is this
> > > > > Fixing the issue?
> > > > >
> > > > > [main] [dpdk.org] $ git diff
> > > > > diff --git a/config/arm/meson.build b/config/arm/meson.build index
> > > > > 00bc4610a..ef65b4bb6 100644
> > > > > --- a/config/arm/meson.build
> > > > > +++ b/config/arm/meson.build
> > > > > @@ -96,15 +96,18 @@ implementer_cavium = {
> > > > > ],
> > > > > 'part_number_config': {
> > > > > '0xa1': {
> > > > > - 'machine_args': ['-mcpu=thunderxt88'],
> > > > > + 'machine_args':
> > > > > ['-march=armv8.1-a+crc+crypto+lse',
> > > > > + '-mcpu=thunderxt88'],
> > > > > 'flags': flags_part_number_thunderx
> > > > > },
> > > > > '0xa2': {
> > > > > - 'machine_args': ['-mcpu=thunderxt81'],
> > > > > + 'machine_args':
> > > > > ['-march=armv8.1-a+crc+crypto+lse',
> > > > > + '-mcpu=thunderxt81'],
> > > > > 'flags': flags_part_number_thunderx
> > > > > },
> > > > > '0xa3': {
> > > > > - 'machine_args': ['-mcpu=thunderxt83'],
> > > > > + 'machine_args':
> > > > > ['-march=armv8.1-a+crc+crypto+lse',
> > > > > + '-mcpu=thunderxt83'],
> > > > > 'flags': flags_part_number_thunderx
> > > > > },
> > > > > '0xaf': {
> > > > >
> > > >
> > > > Hi Jerin,
> > > >
> > > > The patch doesn't work. Build failed at link stage.
> > > > I used gcc 8.4 and tried build on thunderxt88.
> > >
> > >
> > > Hi Ruifeng,
> > >
> > > I talked to compiler experts here in Marvell. It looks like compiler
> > > issue, As a
> > > workaround couple of these could try:
> > > 1) Reduce the external libraries linked to the application like mlx5 etc
> >
> > I tried building with lots of drivers disabled. Not yet able to get a
> > successful build.
> >
> > > 2) Add -mcmodel=large flag will fix "relocation truncated to fit" issue as
> > > testing purpose as we are not sure about the implication of this flag.
> >
> > Looks like this flag is not supported by gcc 8.4 that I am using.
> >
> > One thing we can do to overcome the build failure is to switch to default /
> > release build in OBS CI.
> > OBS CI is running native build, so it could hit this issue when CI job is
> > scheduled to thunderxt88 infrastructure.
> > I think we should change to do release build (-Dmachine=default) which is
> > more suitable for generic CI verification.
> > As I checked, release build can pass on my thunderxt88 platform.
> >
> > What do you think?
>
> Hi,
>
> I've done as suggested, and it seems to fare better, thank you.
>
> There is still a build error on CentOS 7, not sure if it is related
That ICE is a known issue from the compiler. Please use gcc 4.8 at
least for arm64 work.
Similar issue[1] present in some 5.2.1, 6.0 version too.
[1]
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67143
> though:
>
> [ 555s] ../drivers/event/octeontx2/otx2_tim_worker.c: In function
> 'otx2_tim_arm_tmo_tick_burst_mod':
> [ 555s] ../drivers/event/octeontx2/otx2_tim_worker.c:154:18: error: could
> not split insn
> [ 555s] struct rte_event_timer **tim, \
> [ 555s] ^
> [ 555s] ../drivers/event/octeontx2/otx2_tim_evdev.h:208:1: note: in
> expansion of macro 'FP'
> [ 555s] FP(mod, 0, 0, 0, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_DFB) \
> [ 555s] ^
> [ 555s] ../drivers/event/octeontx2/otx2_tim_worker.c:161:1: note: in
> expansion of macro 'TIM_ARM_TMO_FASTPATH_MODES'
> [ 555s] TIM_ARM_TMO_FASTPATH_MODES
> [ 555s] ^
> [ 555s] (insn 252 250 255 (parallel [
> [ 555s] (set (reg:DI 1 x1 [orig:230 D.17092 ] [230])
> [ 555s] (mem/v:DI (reg/f:DI 10 x10 [orig:229 D.17094 ]
> [229]) [-1 S8 A64]))
> [ 555s] (set (mem/v:DI (reg/f:DI 10 x10 [orig:229 D.17094 ]
> [229]) [-1 S8 A64])
> [ 555s] (unspec_volatile:DI [
> [ 555s] (plus:DI (mem/v:DI (reg/f:DI 10 x10
> [orig:229 D.17094 ] [229]) [-1 S8 A64])
> [ 555s] (const_int 1099511627776
> [0x10000000000]))
> [ 555s] (const_int 2 [0x2])
> [ 555s] ] UNSPECV_ATOMIC_OP))
> [ 555s] (clobber (reg:CC 66 cc))
> [ 555s] (clobber (reg:DI 4 x4))
> [ 555s] (clobber (reg:SI 3 x3))
> [ 555s] ]) ../drivers/event/octeontx2/otx2_tim_worker.h:81 1832
> {atomic_fetch_adddi}
> [ 555s] (expr_list:REG_UNUSED (reg:CC 66 cc)
> [ 555s] (expr_list:REG_UNUSED (reg:DI 4 x4)
> [ 555s] (expr_list:REG_UNUSED (reg:SI 3 x3)
> [ 555s] (nil)))))
> [ 555s] ../drivers/event/octeontx2/otx2_tim_worker.c:154:18: internal
> compiler error: in final_scan_insn, at final.c:2897
> [ 555s] struct rte_event_timer **tim, \
> [ 555s] ^
> [ 555s] ../drivers/event/octeontx2/otx2_tim_evdev.h:208:1: note: in
> expansion of macro 'FP'
> [ 555s] FP(mod, 0, 0, 0, OTX2_TIM_BKT_MOD | OTX2_TIM_ENA_DFB) \
> [ 555s] ^
> [ 555s] ../drivers/event/octeontx2/otx2_tim_worker.c:161:1: note: in
> expansion of macro 'TIM_ARM_TMO_FASTPATH_MODES'
> [ 555s] TIM_ARM_TMO_FASTPATH_MODES
> [ 555s] ^
> [ 555s] Please submit a full bug report,
> [ 555s] with preprocessed source if appropriate.
> [ 555s] See <http://bugzilla.redhat.com/bugzilla> for instructions.
> [ 555s] {standard input}: Assembler messages:
> [ 555s] {standard input}: Error: open CFI at the end of file; missing
> .cfi_endproc directive
> [ 555s] Preprocessed source stored into /tmp/ccpVQUdT.out file, please
> attach this to your bugreport.
>
> --
> Kind regards,
> Luca Boccassi
