Re: GCC GSoC 2021 - Static analyzer project

2021-01-23 Thread Adharsh Kamath via Gcc
On Fri, Jan 22, 2021 at 9:32 PM David Malcolm  wrote:
>
> On Fri, 2021-01-22 at 20:46 +0530, Adharsh Kamath wrote:
> > Hi David. Thank you for the reply.
> > On Tue, Jan 19, 2021 at 2:12 AM David Malcolm 
> > wrote:
> > > On Thu, 2021-01-14 at 10:45 +0530, Adharsh Kamath wrote:
> > > > Hello,
> > > > I came across the list of possible project ideas for GSoC 2021
> > > > and
> > > > I'd
> > > > like to contribute to the project regarding the static analysis
> > > > pass
> > > > in GCC.
> > > > How can I get started with this project?
> > >
> > > Hi Adharsh
> > >
> > > Sorry about the delay in responding to your email.
> > >
> > > Thanks for your interest in the static analysis pass.
> > >
> > > Some ideas on getting started with GCC are here:
> > >   https://gcc.gnu.org/wiki/SummerOfCode#Before_you_apply
> > >
> > > The analyzer has its own wiki page here:
> > >   https://gcc.gnu.org/wiki/DavidMalcolm/StaticAnalyzer
> >
> > I examined the analyzer dumps for a few programs. I also read the
> > documentation on the internals of
> > the static analyzer and I've understood the basics of how the
> > analyzer works.
>
> Excellent.  Building GCC from source and stepping through it in the
> debugger would be good next steps.  You'll need plenty of disk space.
>  "run_checkers" is a good breakpoint to set if you're looking for the
> entrypoint to the analyzer.

Yes, I'll try this.

> > > I've actually already implemented some of the ideas that were on
> > > the
> > > GSoC wiki page myself since last summer, so I've updated that page
> > > accordingly:
> > >
> > > https://gcc.gnu.org/wiki/SummerOfCode?action=diff&rev2=187&rev1=184
> > > I've added the idea of SARIF ( https://sarifweb.azurewebsites.net/
> > > ) as
> > > an output format for the static analyzer (and indeed, for the GCC
> > > diagnostics subsystem as a whole).
> > >
> > > Do any of the ideas on the page look appealing to you?  I'm open to
> > > other ideas you may have relating to the analyzer, or indeed to gcc
> > > diagnostics.
> >
> > Yes. Making a plugin for the Linux kernel seems very interesting to
> > me.
> > I'd also like to extend support for C++ but I'm not sure if both
> > ideas would be
> > possible, given the time constraints.
>
> I think that picking just one would be better than trying to do both.
>
> > How do I start with the plugin for
> > the Linux kernel?
>
> I added plugin support to the analyzer in:
> https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=66dde7bc64b75d4a338266333c9c490b12d49825
>
> There's an example plugin in that patch.  The kernel source tree
> already has some plugins, so hopefully those together give some
> pointers on how to write a "hello world" analyzer plugin that runs as
> part of the kernel build, which would be another next step, I guess.

I will go through the plugin in the kernel source tree and try to write a simple
analyzer plugin.

> Unfortunately I'm not a Linux kernel developer, so I don't have deep
> knowledge of what checks would be useful and the subtle details that
> are likely to be necessary.  I'll try to reach out internally within
> Red Hat - we have plenty of kernel developers here.
>
> Some ideas:
> * detecting code paths that acquire a lock but then fail to release it
> * detecting code paths that disable interrupts and then fail to re-
> enable them
> * detecting mixups between user-space pointers and kernel-space
> pointers
>
> The kernel has its own checker called "smatch" which may give other
> ideas for warnings.
>
> The state machine checker in the analyzer takes its inspiration from
> the Stanford "MC" checker (among other places, such as typestate),
> which has been used to implement warnings for the Linux kernel, albeit
> some very old versions of the kernel.
>
> See::
>   * "How to write system-specific, static checkers in Metal" (Benjamin
> Chelf, Dawson R Engler, Seth Hallem), from 2002
>   * "Checking system rules using system-specific, programmer-written
> compiler extensions" Proceedings of Operating Systems Design and
> Implementation (OSDI), September 2000. D. Engler, B. Chelf, A. Chou,
> and S. Hallem.
>   * "Using Programmer-Written Compiler Extensions to Catch Security
> Holes" (Ken Ashcraft, Dawson Engler) from 2002
>
> These are working on 20-year-old in-kernel APIs that might be obsolete
> now, but they have examples of interrupt checking, and user-space vs
> kernel-space pointer checking.
>
> Focusing on error-handling paths in driver code might be best.
>
> Does this answer your questions?

Yes. Thank you very much for these resources. I will go through them.

Adharsh


Re: Building gcc for C and C++ with a custom glibc

2021-01-23 Thread Tom Honermann via Gcc

On 1/17/21 4:08 PM, H.J. Lu wrote:

On Sun, Jan 17, 2021 at 1:06 PM Tom Honermann via Gcc  wrote:

Hi all.  I've been trying to build a custom gcc (trunk) with a custom
glibc (trunk) with support for C and C++ on x86_64 Linux and have so far
been unsuccessful at identifying a sequence of configure/make
invocations that completes successfully.  I'm not trying to build a
cross compiler.

The scenario I'm trying to satisfy is testing some changes to gcc, and
additional changes to libstdc++ that have new autoconf and header
dependencies on the presence of new functions in existing glibc headers.

The glibc installation I'm trying to use was built with:

 mkdir glibc-build
 cd glibc-build
 ../glibc/configure \
  CC=gcc \
  CXX=g++ \
  --prefix /.../glibc
 make && make install

For gcc, I've tried numerous variants of the following:

 mkdir gcc-build
 cd gcc-build
 ../gcc/configure \
  CC=gcc \
  CXX=g++ \
  --prefix /.../gcc \
  --disable-multilib \
  --enable-languages=c,c++ \
  --enable-checking=release \
  --disable-bootstrap \
  --disable-lto

Things I've tried include setting CFLAGS, CXXFLAGS, and LDFLAGS to
specify additional glibc paths, to specify alternate paths (via
-nostdinc/-nostdinc++), setting LIBRARY_PATH and CPATH, passing
--with-sysroot, passing --with-headers and --with-libs, passing
--with-native-system-header-dir, some of those in conjunction with
removing --disable-bootstrap, and wrapping gcc in a script that attempts
to substitute certain include paths.

The problem I'm having is that, in every attempt, the glibc headers and
libraries from under /usr end up getting used instead of the custom
glibc ones at some point leading to build failures.

Does anyone have a recipe available for doing this?

Try scripts/build-many-glibcs.py in glibc source.

Thank you for the suggestion.  I studied the script and tried using it, 
but it also encountered errors during the 2nd stage gcc build.  I didn't 
delve into those errors.


This is what ended up working for me:

   mkdir -p "$BINUTILS_BUILD_DIR"
   cd "$BINUTILS_BUILD_DIR"
   "../$BINUTILS_SOURCE_DIR/configure" \
  CC=gcc \
  CXX=g++ \
  --prefix "$INSTALL_DIR" \
  --with-sysroot="$INSTALL_DIR" \
  --disable-multilib \
   ;
   make -j 4 && make install
   cd -

   cd "$LINUX_SOURCE_DIR"
   make INSTALL_HDR_PATH="$INSTALL_DIR" headers_install
   cd -

   mkdir -p "$GLIBC_BUILD_DIR"
   cd "$GLIBC_BUILD_DIR"
   "../$GLIBC_SOURCE_DIR/configure" \
  CC=gcc \
  CXX=g++ \
  --prefix "$INSTALL_DIR" \
  --disable-multilib \
  --with-headers="$INSTALL_DIR/include" \
   ;
   make -j 4 && make install
   cd -

   # Fixup paths in libc.so and libm.so.  I wasn't able to find a
   combination of
   # 'configure --prefix' and 'make prefix=... DESTIDIR=...' that did
   the right thing.
   cp "$INSTALL_DIR/lib/libc.so" "$INSTALL_DIR/lib/libc.so.orig"
   sed -e "s%$INSTALL_DIR%%g" "$INSTALL_DIR/lib/libc.so.orig" >
   "$INSTALL_DIR/lib/libc.so"
   rm -f "$INSTALL_DIR/lib/libc.so.orig"
   cp "$INSTALL_DIR/lib/libm.so" "$INSTALL_DIR/lib/libm.so.orig"
   sed -e "s%$INSTALL_DIR%%g" "$INSTALL_DIR/lib/libm.so.orig" >
   "$INSTALL_DIR/lib/libm.so"
   rm -f "$INSTALL_DIR/lib/libm.so.orig"

   mkdir -p "$GCC_BUILD_DIR"
   cd "$GCC_BUILD_DIR"
   "../$GCC_SOURCE_DIR/configure" \
  CC=gcc \
  CXX=g++ \
  --prefix "$INSTALL_DIR" \
  --with-sysroot="$INSTALL_DIR" \
  --with-native-system-header-dir="/include" \
  --enable-languages=c,c++ \
  --disable-bootstrap \
  --disable-multilib \
  --enable-checking=release \
  --disable-lto \
   ;
   make -j 4 && make install
   cd -

Tom.



gcc-10-20210123 is now available

2021-01-23 Thread GCC Administrator via Gcc
Snapshot gcc-10-20210123 is now available on
  https://gcc.gnu.org/pub/gcc/snapshots/10-20210123/
and on various mirrors, see http://gcc.gnu.org/mirrors.html for details.

This snapshot has been generated from the GCC 10 git branch
with the following options: git://gcc.gnu.org/git/gcc.git branch 
releases/gcc-10 revision 0184527ae9f81ef3865703b43212963b95ae174d

You'll find:

 gcc-10-20210123.tar.xz   Complete GCC

  SHA256=47cbfb3b76dab5eb21c1d97b80c3b765448a56c332f074d3014b0e628b539ab2
  SHA1=da0a2251937b8b497bcad541869ebf66e935eba0

Diffs from 10-20210116 are available in the diffs/ subdirectory.

When a particular snapshot is ready for public consumption the LATEST-10
link is updated and a message is sent to the gcc list.  Please do not use
a snapshot before it has been announced that way.