Re: [RFC] analyzer: allocation size warning
The checker reaches region-model.cc#3083 in my patch with the impl_region_model_context on the 'after' node of create_buffer() but then discards the warning inside impl_region_model_context::warn because m_stmt is null. Even if m_stmt were not be NULL at the 'after' node, my warning would be emitted before the return edge was taken and thus be wrongly indented like shown below: /path/to/.c:10:16: warning: Allocated buffer size is not a multiple of the pointee's size [CWE-131] [-Wanalyzer-allocation-size] 10 | int *buf = create_buffer(42); |^ ‘main’: events 1-2 | |9 | int main (int argc, char **argv) { | | ^~~~ | | | | | (1) entry to ‘main’ | 10 | int *buf = create_buffer(42); | |~ | || | |(2) calling ‘create_buffer’ from ‘main’ | +--> ‘create_buffer’: events 3-4 | |4 | void *create_buffer(int n) | | ^ | | | | | (3) entry to ‘create_buffer’ |5 | { |6 | return malloc(n); | |~ | || | |(4) allocated 42 bytes here | ‘main’: event 5 | | 10 | int *buf = create_buffer(42); | |^ | || | |(5) Assigned to ‘int *’ here | The correct warning should be: /path/to/.c:10:16: warning: Allocated buffer size is not a multiple of the pointee's size [CWE-131] [-Wanalyzer-allocation-size] 10 | int *buf = create_buffer(42); |^ ‘main’: events 1-2 | |9 | int main (int argc, char **argv) { | | ^~~~ | | | | | (1) entry to ‘main’ | 10 | int *buf = create_buffer(42); | |~ | || | |(2) calling ‘create_buffer’ from ‘main’ | +> ‘create_buffer’: events 3-4 | |4 | void *create_buffer(int n) | | ^ | | | | | (3) entry to ‘create_buffer’ |5 | { |6 | return malloc(n); | |~ | || | |(4) allocated 42 bytes here | ‘main’: event 5 <--+ | | 10 | int *buf = create_buffer(42); | |^ | || | |(5) Assigned to ‘int *’ here | For that, the return edge has to be visited to be part of the emission_path. This is currently not the case as the assignment of the to the caller lhs is handled inside pop_frame, which is transitively called from program_state::on_edge of the 'after' node of the callee. I tried to defer the set_value(caller lhs, ) call to the 'before' node after the return edge but failed to do elegantly. My last try is in the patch commented out with // FIXME. My main problem is that I can not pop the frame and later get the return value easily. Deferring the whole pop_frame to the before node breaks the assumptions inside exploded_graph::get_or_create_node. I don't know what's the best/elegant way of solving this. Is a solution to attach the return svalue to the return edge and then use it later in the PK_BEFORE_SUPERNODE? Signed-off-by: Tim Lange --- gcc/analyzer/analyzer.opt | 4 + gcc/analyzer/checker-path.cc | 12 +- gcc/analyzer/checker-path.h | 2 +- gcc/analyzer/engine.cc| 12 + gcc/analyzer/pending-diagnostic.h | 21 ++ gcc/analyzer/region-model.cc | 322 ++ gcc/analyzer/region-model.h | 4 + .../gcc.dg/analyzer/allocation-size-1.c | 63 .../gcc.dg/analyzer/allocation-size-2.c | 44 +++ .../gcc.dg/analyzer/allocation-size-3.c | 48 +++ .../gcc.dg/analyzer/allocation-size-4.c | 92 + gcc/testsuite/gcc.dg/analyzer/attr-malloc-6.c | 2 + gcc/testsuite/gcc.dg/analyzer/malloc-4.c | 2 +- gcc/testsuite/gcc.dg/analyzer/pr96639.c | 2 + 14 files changed, 627 insertions(+), 3 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/analyzer/allocation-size-1.c create mode 100644 gcc/testsuite/gcc.dg/analyzer/allocation-size-2.c create mode 100644 gcc/testsuite/gcc.dg/analyzer/allocation-size-3.c create mode 100644 gcc/testsuite/gcc.dg/analyzer/allocation-size-4.
Re: [RFC] analyzer: allocation size warning
On Wed, 2022-06-22 at 16:57 +0200, Tim Lange wrote: > The checker reaches region-model.cc#3083 in my patch with the > impl_region_model_context > on the 'after' node of create_buffer() but then discards the warning > inside > impl_region_model_context::warn because m_stmt is null. Even if m_stmt > were > not be NULL at the 'after' node, my warning would be emitted before the > return edge was taken and thus be wrongly indented like shown below: > /path/to/.c:10:16: warning: Allocated buffer size is not a multiple of > the pointee's size [CWE-131] [-Wanalyzer-allocation-size] > 10 | int *buf = create_buffer(42); > | ^ > ‘main’: events 1-2 > | > | 9 | int main (int argc, char **argv) { > | | ^~~~ > | | | > | | (1) entry to ‘main’ > | 10 | int *buf = create_buffer(42); > | | ~ > | | | > | | (2) calling ‘create_buffer’ from ‘main’ > | > +--> ‘create_buffer’: events 3-4 > | > | 4 | void *create_buffer(int n) > | | ^ > | | | > | | (3) entry to ‘create_buffer’ > | 5 | { > | 6 | return malloc(n); > | | ~ > | | | > | | (4) allocated 42 bytes here > | > ‘main’: event 5 > | > | 10 | int *buf = create_buffer(42); > | | ^ > | | | > | | (5) Assigned to ‘int *’ here > | > > The correct warning should be: > /path/to/.c:10:16: warning: Allocated buffer size is not a multiple of > the pointee's size [CWE-131] [-Wanalyzer-allocation-size] > 10 | int *buf = create_buffer(42); > | ^ > ‘main’: events 1-2 > | > | 9 | int main (int argc, char **argv) { > | | ^~~~ > | | | > | | (1) entry to ‘main’ > | 10 | int *buf = create_buffer(42); > | | ~ > | | | > | | (2) calling ‘create_buffer’ from ‘main’ > | > +> ‘create_buffer’: events 3-4 > | > | 4 | void *create_buffer(int n) > | | ^ > | | | > | | (3) entry to ‘create_buffer’ > | 5 | { > | 6 | return malloc(n); > | | ~ > | | | > | | (4) allocated 42 bytes here > | > ‘main’: event 5 <--+ > | > | 10 | int *buf = create_buffer(42); > | | ^ > | | | > | | (5) Assigned to ‘int *’ here > | > For that, the return edge has to be visited to be part of the > emission_path. > This is currently not the case as the assignment of the > to > the caller lhs is handled inside pop_frame, which is transitively > called > from program_state::on_edge of the 'after' node of the callee. > I tried to defer the set_value(caller lhs, ) call to the > 'before' node after the return edge but failed to do elegantly. My last > try > is in the patch commented out with // FIXME. > My main problem is that I can not pop the frame and later get the > return value easily. Deferring the whole pop_frame to the before node > breaks the assumptions inside exploded_graph::get_or_create_node. > > I don't know what's the best/elegant way of solving this. Is a solution > to > attach the return svalue to the return edge and then use it later in > the > PK_BEFORE_SUPERNODE? The ctxt is created here: #5 0x012f5856 in ana::program_state::on_edge (this=this@entry=0x7fffc8c0, eg=..., enode=enode@entry=0x2d8d970, succ=succ@entry=0x2d0e590, uncertainty=uncertainty@entry=0x7fffc990) at ../../src/gcc/analyzer/program-state.cc:1035 1035 if (!m_region_model->maybe_update_for_edge (*succ, (gdb) list 1030 impl_region_model_context ctxt (eg, enode, 1031 &enode->get_state (), 1032 this, 1033 uncertainty, NULL, 1034 last_stmt); 1035 if (!m_region_model->maybe_update_for_edge (*succ, 1036 last_stmt, 1037 &ctxt, NULL)) I tried another approach: to provide a custom stmt_finder for this ctxt, which uses the "re
Re: [MRISC32] Not getting scaled index addressing in loops
On Fri, May 27, 2022 at 11:52 PM m wrote: > > Hello! > > I maintain a fork of GCC which adds support for my custom CPU ISA, > MRISC32 (the machine description can be found here: > https://github.com/mrisc32/gcc-mrisc32/tree/mbitsnbites/mrisc32/gcc/config/mrisc32 > ). > > I recently discovered that scaled index addressing (i.e. MEM[base + > index * scale]) does not work inside loops, but I have not been able to > figure out why. > > I believe that I have all the plumbing in the MD that's required > (MAX_REGS_PER_ADDRESS, REGNO_OK_FOR_BASE_P, REGNO_OK_FOR_INDEX_P, etc), > and I have verified that scaled index addressing is used in trivial > cases like this: > > charcarray[100]; > shortsarray[100]; > intiarray[100]; > voidsingle_element(intidx, intvalue) { > carray[idx] = value; // OK > sarray[idx] = value; // OK > iarray[idx] = value; // OK > } > > ...which produces the expected machine code similar to this: > > stbr2, [r3, r1] // OK > sthr2, [r3, r1*2] // OK > stwr2, [r3, r1*4] // OK > > However, when the array assignment happens inside a loop, only the char > version uses index addressing. The other sizes (short and int) will be > transformed into code where the addresses are stored in registers that > are incremented by +2 and +4 respectively. > > voidloop(void) { > for(intidx = 0; idx < 100; ++idx) { > carray[idx] = idx; // OK > sarray[idx] = idx; // BAD > iarray[idx] = idx; // BAD > } > } ...which produces: > .L4: > sthr1, [r3] // BAD > stwr1, [r2] // BAD > stbr1, [r5, r1] // OK > addr1, r1, #1 > sner4, r1, #100 > addr3, r3, #2 // (BAD) > addr2, r2, #4 // (BAD) > bsr4, .L4 > > I would expect scaled index addressing to be used in loops too, just as > is done for AArch64 for instance. I have dug around in the machine > description, but I can't really figure out what's wrong. > > For reference, here is the same code in Compiler Explorer, including the > code generated for AArch64 for comparison: https://godbolt.org/z/drzfjsxf7 > > Passing -da (dump RTL all) to gcc, I can see that the decision to not > use index addressing has been made already in *.253r.expand. The problem is your cost model for the indexing is incorrect; IV-OPTs uses TARGET_ADDRESS_COST to figure out the cost of each case. So if you don't have that implemented, then the default one is used and that will be incorrect in many cases. You can find IV-OPTs costs and such by using the ivopts dump: -fdump-tree-ivopts-details . Thanks, Andrew Pinski > > Does anyone have any hints about what could be wrong and where I should > start looking? > > Regards, > >Marcus >
Re: remove intl/ directory?
Hi Bruno, +1 on the C reasons for removing intl. (however, once we have a rough working patch, it would still need buy-in from GDB + binutils) > On 21 Jun 2022, at 03:05, Bruno Haible wrote: >> So, indeed, part of this is quite straight forward - we can amend the >> Makefile.def >> to specify that GCC should use gettext-runtime (it will be used if the >> directory is >> present, otherwise there will be no intl support). >> >> The tricky part is that we need to use the runtime ‘uninstalled’, and here >> is where >> intl is helpful - it provides a ‘config.intl’ that can be sourced via >> gettext-sister.m4 >> to provide the neccessary configure input to directories that want to use >> intl. >> >> I have hacked a change to gettext-sister.m4 that fishes the same information >> out >> of gettext-runtime/Makefile (as configured in $build) - obviously this is >> going to be >> fragile w.r.t different versions of gettext-runtime (so I am not suggesting >> this is a >> viable patch) - simply something to illustrate what needs to be figured out. >> >> So - the changes are in Makefile.def and config/gettext-sister.m4 (the patch >> includes >> the regenerated files for convenience of use). >> >> I tried this with gettext-0.21 on macOS 10.15 and, AFAICT, it DTRT - but >> needs work >> to resolve the main point above. > > Excellent! Glad to see that you are going ahead so quickly. > > I have now added the necessary support for this "uninstalled" situation from > the gettext side: > https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commitdiff;h=6f048e30a88282ed6712ce8d6000302fd287daad > This is in a form that is maintainable from gettext's side. > > I believe that with this, you can change these lines from gettext-sister.m4 > > BUILD_INCLUDED_LIBINTL=`grep BUILD_INCLUDED_LIBINTL > ../gettext-runtime/Makefile|sed s/BUILD_INCLUDED_LIBINTL\ =\ //` > USE_NLS=`grep USE_NLS ../gettext-runtime/Makefile|sed s/USE_NLS\ =\ //` > LIBICONV=`grep ^LIBICONV ../gettext-runtime/Makefile|sed s/LIBICONV\ =\ //` > INTL_MACOSX_LIBS=`grep ^INTL_MACOSX_LIBS ../gettext-runtime/Makefile|sed > s/INTL_MACOSX_LIBS\ =\ //` > XGETTEXT=`grep XGETTEXT ../gettext-runtime/Makefile|sed s/XGETTEXT\ =\ //` > GMSGFMT=`grep GMSGFMT ../gettext-runtime/Makefile|sed s/GMSGFMT\ =\ //` > POSUB=`grep POSUB ../gettext-runtime/Makefile|sed s/POSUB\ =\ //` > LIBINTL="\${top_builddir}/../gettext-runtime/intl/.libs/libintl.a $LIBICONV > $INTL_MACOSX_LIBS" > INCINTL="-I\${top_builddir}/../gettext-runtime/intl" > > roughly to this: > > relative_builddir='${top_builddir}/../gettext-runtime' > . ${top_builddir}/../gettext-runtime/uninstalled-config.sh Yes ( # We can use an in-tree build of libintl. if test -f ifelse([$1],,[../gettext-runtime],[$1])/uninstalled-config.sh; then relative_builddir='ifelse([$1],,[${top_builddir}/..],[$1]/..)/gettext-runtime' . ifelse([$1],,[../gettext-runtime],[$1])/uninstalled-config.sh elif test -f ifelse([$1],,[../intl],[$1])/config.intl; then . ifelse([$1],,[../intl],[$1])/config.intl fi ) and it works ... … although now I see some configure warnings about not being able to access build-aux (which I do not recall seeing with the previous hack - but that could be just bad memory ;) ) - FWIW this following snippet would be just as broken on macOS as other noted platforms - it would need auto-foo-provided shared lib extension - or the equivalent to be used. … is there any reason that all platforms with non-’so’ suffixes would not work with that change? +# LIBINTL is a set of compiler options, to use when linking without libtool, +# that ensures that the library that contains the *gettext() family of functions +# gets found. +if test @USE_INCLUDED_LIBINTL@ = yes; then + if test '@ENABLE_SHARED@' = yes; then +# NB: This case is not supported on AIX and HP-UX. +LIBINTL="${relative_builddir}/intl/.libs/libintl.so -Wl,-rpath,${relative_builddir}/intl/.libs @LIBICONV@ @INTL_MACOSX_LIBS@" + else +LIBINTL="${relative_builddir}/intl/.libs/libintl.a @LIBICONV@ @INTL_MACOSX_LIBS@" + fi +else + # The functionality is provided by libc. + LIBINTL= +fi > There is also a GCC specific quirk, that I upstreamed into GNU gettext: > https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commitdiff;h=fdc2bd236a6a62b477c1fca4205df10b0e64266b IMHO we need to fix this ^ in GCC config - since gettext-runtime accepts “—with-pic” we should amend the GCC configury to pass —with-pic [to GMP et. al. as well, currently to build in-tree with host-shared, needs a manual —with-pic on the configure too] I added a change that looks at the [host-shared] flag in the top-level configure, it just needs to be moved and used to add the —with-pic flags where needed (where ‘just’ is more than 5mins work, and needs testing widely …) >> P.S. I am slighty surprised that configuring with —disable-java does not >> appear to stop >> the on-screen popup appearing that tells me I need to install Java t
Connecting From LinkedIn
-- Hello from LinkedIn, Is your email still active?
Re: remove intl/ directory?
Iain Sandoe wrote: > Yes ( > # We can use an in-tree build of libintl. > if test -f ifelse([$1],,[../gettext-runtime],[$1])/uninstalled-config.sh; > then > > relative_builddir='ifelse([$1],,[${top_builddir}/..],[$1]/..)/gettext-runtime' > . ifelse([$1],,[../gettext-runtime],[$1])/uninstalled-config.sh > elif test -f ifelse([$1],,[../intl],[$1])/config.intl; then > . ifelse([$1],,[../intl],[$1])/config.intl > fi > ) > and it works ... Good! > … although now I see some configure warnings about not being able to access > build-aux (which I do not recall seeing with the previous hack - but that > could be just bad memory ;) ) You can get warnings if you _move_ the gettext-runtime directory so that it becomes a sibling directory of 'gcc'. You should *not* get warnings if you create a symlink, sibling of the 'gcc' directory, to the gettext-20220620/gettext-runtime/ directory. > FWIW this following snippet would be just as broken on macOS as other noted > platforms - it would need auto-foo-provided shared lib extension - or the > equivalent to be used. > … is there any reason that all platforms with non-’so’ suffixes would not > work with that change? On macOS (with .dylib instead of .so) it would probably work. However, AIX and HP-UX will not work, because (as I understand it) if you want to have a binary, say cc1, which depends on libintl, then - the cc1 that accesses /usr/local/lib/libintl.$suffix and - the cc1 that accesses /home/user/build/gcc-snap/gettext-runtime/intl/.libs/libintl.$suffix must necessarily be different. You cannot just install the second one in public locations, because it will have the wrong shared library filename hardcoded into it. This is why on these systems, libtool has to rebuild executables during "make install". Anyway, you said that for GCC, the important case is to build libintl as a static (non-shared) library. > > There is also a GCC specific quirk, that I upstreamed into GNU gettext: > > https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commitdiff;h=fdc2bd236a6a62b477c1fca4205df10b0e64266b > > IMHO we need to fix this ^ in GCC config - since gettext-runtime accepts > “—with-pic” we should amend the GCC configury to pass —with-pic [to GMP et. > al. as well, currently to build in-tree with host-shared, needs a manual > —with-pic on the configure too] Indeed, the option '--with-pic' (from libtool) has the same effect as '--enable-host-shared'. If you can arrange to pass '--with-pic' instead of '--enable-host-shared', I can revert the addition of the option '--enable-host-shared' in gettext-runtime/intl from two days ago. > I think that we now need to deal with the GCC-side of the configury … > > 1) add logic [like GMP et. al.] to specify an external source of the library > (when there is no-in-tree source present) Are you aware that gettext.m4 already introduces the configure options --with-libintl-prefix[=DIR] search for libintl in DIR/include and DIR/lib --without-libintl-prefix don't search for libintl in includedir and libdir ? Bruno
Re: [MRISC32] Not getting scaled index addressing in loops
On 2022-06-22, Andrew Pinski wrote: On Fri, May 27, 2022 at 11:52 PM m wrote: Hello! I maintain a fork of GCC which adds support for my custom CPU ISA, MRISC32 (the machine description can be found here: https://github.com/mrisc32/gcc-mrisc32/tree/mbitsnbites/mrisc32/gcc/config/mrisc32 ). I recently discovered that scaled index addressing (i.e. MEM[base + index * scale]) does not work inside loops, but I have not been able to figure out why. I believe that I have all the plumbing in the MD that's required (MAX_REGS_PER_ADDRESS, REGNO_OK_FOR_BASE_P, REGNO_OK_FOR_INDEX_P, etc), and I have verified that scaled index addressing is used in trivial cases like this: charcarray[100]; shortsarray[100]; intiarray[100]; voidsingle_element(intidx, intvalue) { carray[idx] = value; // OK sarray[idx] = value; // OK iarray[idx] = value; // OK } ...which produces the expected machine code similar to this: stbr2, [r3, r1] // OK sthr2, [r3, r1*2] // OK stwr2, [r3, r1*4] // OK However, when the array assignment happens inside a loop, only the char version uses index addressing. The other sizes (short and int) will be transformed into code where the addresses are stored in registers that are incremented by +2 and +4 respectively. voidloop(void) { for(intidx = 0; idx < 100; ++idx) { carray[idx] = idx; // OK sarray[idx] = idx; // BAD iarray[idx] = idx; // BAD } } ...which produces: .L4: sthr1, [r3] // BAD stwr1, [r2] // BAD stbr1, [r5, r1] // OK addr1, r1, #1 sner4, r1, #100 addr3, r3, #2 // (BAD) addr2, r2, #4 // (BAD) bsr4, .L4 I would expect scaled index addressing to be used in loops too, just as is done for AArch64 for instance. I have dug around in the machine description, but I can't really figure out what's wrong. For reference, here is the same code in Compiler Explorer, including the code generated for AArch64 for comparison: https://godbolt.org/z/drzfjsxf7 Passing -da (dump RTL all) to gcc, I can see that the decision to not use index addressing has been made already in *.253r.expand. The problem is your cost model for the indexing is incorrect; IV-OPTs uses TARGET_ADDRESS_COST to figure out the cost of each case. So if you don't have that implemented, then the default one is used and that will be incorrect in many cases. You can find IV-OPTs costs and such by using the ivopts dump: -fdump-tree-ivopts-details . Thanks, Andrew Pinski Thank you Andrew! I added a TARGET_ADDRESS_COST implementation that just returns zero, as a test, and sure enough scaled indexed addressing was used. Now I will just have to figure out a more accurate implementation for my architecture. Regards, Marcus Does anyone have any hints about what could be wrong and where I should start looking? Regards, Marcus
Re: remove intl/ directory?
Hi Bruno, > On 23 Jun 2022, at 05:24, Bruno Haible wrote: > > Iain Sandoe wrote: >> … although now I see some configure warnings about not being able to access >> build-aux (which I do not recall seeing with the previous hack - but that >> could be just bad memory ;) ) > > You can get warnings if you _move_ the gettext-runtime directory so that it > becomes a sibling directory of 'gcc'. You should *not* get warnings if you > create a symlink, sibling of the 'gcc' directory, to the > gettext-20220620/gettext-runtime/ directory. I did symlink, and agree it should work - I’ll need to try and repeat when next I have some time. > >> FWIW this following snippet would be just as broken on macOS as other noted >> platforms - it would need auto-foo-provided shared lib extension - or the >> equivalent to be used. >> … is there any reason that all platforms with non-’so’ suffixes would not >> work with that change? > > On macOS (with .dylib instead of .so) it would probably work. > > However, AIX and HP-UX will not work, because (as I understand it) if you want > to have a binary, say cc1, which depends on libintl, then > - the cc1 that accesses /usr/local/lib/libintl.$suffix > and > - the cc1 that accesses > /home/user/build/gcc-snap/gettext-runtime/intl/.libs/libintl.$suffix > must necessarily be different. You cannot just install the second one in > public locations, because it will have the wrong shared library filename > hardcoded into it. This is why on these systems, libtool has to rebuild > executables during "make install". Ah, actually a similar situation might apply to the macOS case, you would either need to build it “@rpath” and install the library in the exe’s dir or build and install it into ‘prefix’ (that puts the full pathname into the dylib, in a similar way to AIX / HP-UX). This is also requires a bit of juggling on macOS (I have patches in flight to make all the runtimes for GCC built with ‘@rpath’ and using embedded rpaths in exe) hopefully for GCC-13 … so let’s quietly forget the shared case for now... > Anyway, you said that for GCC, the important case is to build libintl as a > static (non-shared) library. Yes, in a 1:1 replacement for ‘intl’ that’s the case, we can figure out shared stuff as a follow-on. >> I think that we now need to deal with the GCC-side of the configury … >> >> 1) add logic [like GMP et. al.] to specify an external source of the library >> (when there is no-in-tree source present) > > Are you aware that gettext.m4 already introduces the configure options > --with-libintl-prefix[=DIR] search for libintl in DIR/include and DIR/lib > --without-libintl-prefix don't search for libintl in includedir and > libdir Hmm - the following cases: a) there’s no gettext-runtime in the source tree and the user needs to configure —with-libintl-prefix= b) there is a gettext-runtime in the source tree and the user decides to configure —with-libintl-prefix= (which will be ignored if we take the way the other in-tree builds are handled as ’status quo’ c) there is a gettext-runtime in the source tree and no —with-libintl-prefix= is given (we expect to pick up the in-tree build silently and automatically). … in case (a) we’d need to arrange for the gettext macro to be called in configure, but I don’t think it will play nicely with gettext-sister .. so there’s some work needed here. in case (b) I’m not sure what will happen - will the configure for libintl just point the variables to the install suggested? case (c) works today. cheers Iain