Interest in GSoC Project (offloading to a separate process on the same host)
Hello GCC Community, I am Pranil Dey, a 4th year undergraduate student of the Indian Institute of Technology Kharagpur currently pursuing a Bachelor's Degree in Computer Science and Engineering. I am interested in contributing to the GCC projects under GSoC, specifically the projects : "Offloading to a separate process on the same host" and "Improve nothrow detection in GCC". I have worked on inter process communication in college operating systems projects which have helped me understand more about shared memory, pipes and multithreading concepts. I have also taken compiler design theory and laboratory courses as a part of my institute curriculum. In the lab we designed a Tiny-C compiler (a subset of GCC). Although I have no experience with big projects like GCC, I have built the codebase and am currently trying to understand it further. I would like some pointers to start understanding and contributing to these projects along with any helpful resources/reading material for delving deeper into the relevant topic. Any guidance on proposal formulation will also be appreciated greatly. Thank You, -- Pranil Dey Student of Indian Institute of Technology Kharagpur Dept. of Computer Science and Engineering
Re: [RFC] add regenerate Makefile target
On Thu, 14 Mar 2024 at 19:10, Simon Marchi wrote: > > > > On 2024-03-13 04:02, Christophe Lyon via Gdb wrote: > > Hi! > > > > After recent discussions on IRC and on the lists about maintainer-mode > > and various problems with auto-generated source files, I've written > > this small prototype. > > > > Based on those discussions, I assumed that people generally want to > > update autotools files using a script similar to autoregen.py, which > > takes care of running aclocal, autoheader, automake and autoconf as > > appropriate. > > > > What is currently missing is a "simple" way of regenerating other > > files, which happens normally with --enable-maintainer-mode (which is > > reportedly broken). This patch as a "regenerate" Makefile target > > which can be called to update those files, provided > > --enable-maintainer-mode is used. > > > > I tried this approach with the following workflow for binutils/gdb: > > - run autoregen.py in srcdir > > - cd builddir > > - configure --enable-maintainer-mode > > - make all-bfd all-libiberty regenerate -j1 > > - for gdb: make all -C gdb/data-directory -j1 > > - make all -jXXX > > > > Making 'all' in bfd and libiberty is needed by some XXX-gen host > > programs in opcodes. > > > > The advantage (for instance for CI) is that we can regenerate files at > > -j1, thus avoiding the existing race conditions, and build the rest > > with -j XXX. > > > > Among drawbacks: > > - most sub-components use Makefile.am, but gdb does not: this may make > > maintenance more complex (different rules for different projects) > > - maintaining such ad-hoc "regenerate" rules would require special > > attention from maintainers/reviewers > > - dependency on -all-bfd and all-libiberty is probably not fully > >intuitive, but should not be a problem if the "regenerate" rules > >are used after a full build for instance > > > > Of course Makefile.def/Makefile.tpl would need further cleanup as I > > didn't try to take gcc into account is this patch. > > > > Thoughts? > > My first thought it: why is it a Makefile target, instead of some script > on the side (like autoregen.sh). It would be nice / useful to be > able to it without configuring / building anything. For instance, the > autoregen buildbot job could run it without configuring anything. > Ideally, the buildbot wouldn't maintain its own autoregen.py file on the > side, it would just use whatever is in the repo. Firstly because of what you mention later: some regeneration steps require building host tools first, like the XXX-gen in opcodes. Since the existing Makefiles already contain the rules to autoregen all these files, it seemed natural to me to reuse them, to avoid reinventing the wheel with the risk of introducing new bugs. This involves changes in places where I've never looked at before, so I'd rather reuse as much existing support as possible. For instance, there are the generators in opcodes/, but also things in sim/, bfd/, updates to the docs and potfiles. In gcc, there's also something "unusual" in fixincludes/ and libgfortran/ In fact, I considered also including 'configure', 'Makefile.in', etc... in the 'regenerate' target, it does not seem natural to me to invoke a script on the side, where you have to replicate the behaviour of existing Makefiles, possibly getting out-of-sync when someone forgets to update either Makefile or autoregen.py. What is currently missing is a way to easily regenerate files without having to run a full 'make all' (which currently takes care of calling autoconf & friends to update configure/Makefile.in). But yeah, having to configure before being able to regenerate files is a bit awkward too :-) > > Looking at the rule to re-generate copying.c in gdb for instance: > > # Make copying.c from COPYING > $(srcdir)/copying.c: @MAINTAINER_MODE_TRUE@ $(srcdir)/../COPYING3 > $(srcdir)/copying.awk >awk -f $(srcdir)/copying.awk \ >< $(srcdir)/../COPYING3 > $(srcdir)/copying.tmp >mv $(srcdir)/copying.tmp $(srcdir)/copying.c > > There is nothing in this code that requires having configured the source > tree. This code could for instance be moved to some > generate-copying-c.sh script. generate-copying-c.sh could be called by > an hypothetical autoregen.sh script, as well as the copying.c Makefile > target, if we want to continue supporting the maintainer mode. Wouldn't it be more obscure than now? Currently such build rules are all in the relevant Makefile. You'd have to open several scripts to discover what's involved with updating copying.c > > Much like your regenerate targets, an autoregen.sh script in a given > directory would be responsible to re-generate all the files in this > directory that are generated and checked in git. It would also be > responsible to call any autoregen.sh file in subdirectories. Makefiles already have all that in place :-) Except if you consider that you'd want to ignore timestamps and always regenerate th
Re: Using std types and functions within GCC
> "David" == David Malcolm via Gcc writes: David> For example, there's at David> least one place where I'd have used std::optional, but that's C++14 and David> so unavailable. FWIW, gdb had its own gdb::optional (which was really just a stripped-down copy of the one from libstdc++) to fill exactly this need, at least until we moved to C++17 this year. If you need it you could easily lift it from the gdb repository. Tom
Re: Using std types and functions within GCC
March 15, 2024 at 2:00 PM, "Tom Tromey" wrote: > > > > "David" == David Malcolm via Gcc writes: > > > > David> For example, there's at > David> least one place where I'd have used std::optional, but that's C++14 and > David> so unavailable. > > FWIW, gdb had its own gdb::optional (which was really just a > stripped-down copy of the one from libstdc++) to fill exactly this need, > at least until we moved to C++17 this year. If you need it you could > easily lift it from the gdb repository. > Hi, In the Rust frontend, we are using a single header implementation from https://github.com/TartanLlama/optional, until we can use something from libstdc++. Marc
Re: [RFC] add regenerate Makefile target
On Fri, Mar 15, 2024 at 4:53 AM Christophe Lyon via Gcc wrote: > > On Thu, 14 Mar 2024 at 19:10, Simon Marchi wrote: > > > > > > > > On 2024-03-13 04:02, Christophe Lyon via Gdb wrote: > > > Hi! > > > > > > After recent discussions on IRC and on the lists about maintainer-mode > > > and various problems with auto-generated source files, I've written > > > this small prototype. > > > > > > Based on those discussions, I assumed that people generally want to > > > update autotools files using a script similar to autoregen.py, which > > > takes care of running aclocal, autoheader, automake and autoconf as > > > appropriate. > > > > > > What is currently missing is a "simple" way of regenerating other > > > files, which happens normally with --enable-maintainer-mode (which is > > > reportedly broken). This patch as a "regenerate" Makefile target > > > which can be called to update those files, provided > > > --enable-maintainer-mode is used. > > > > > > I tried this approach with the following workflow for binutils/gdb: > > > - run autoregen.py in srcdir > > > - cd builddir > > > - configure --enable-maintainer-mode > > > - make all-bfd all-libiberty regenerate -j1 > > > - for gdb: make all -C gdb/data-directory -j1 > > > - make all -jXXX > > > > > > Making 'all' in bfd and libiberty is needed by some XXX-gen host > > > programs in opcodes. > > > > > > The advantage (for instance for CI) is that we can regenerate files at > > > -j1, thus avoiding the existing race conditions, and build the rest > > > with -j XXX. > > > > > > Among drawbacks: > > > - most sub-components use Makefile.am, but gdb does not: this may make > > > maintenance more complex (different rules for different projects) > > > - maintaining such ad-hoc "regenerate" rules would require special > > > attention from maintainers/reviewers > > > - dependency on -all-bfd and all-libiberty is probably not fully > > >intuitive, but should not be a problem if the "regenerate" rules > > >are used after a full build for instance > > > > > > Of course Makefile.def/Makefile.tpl would need further cleanup as I > > > didn't try to take gcc into account is this patch. > > > > > > Thoughts? > > > > My first thought it: why is it a Makefile target, instead of some script > > on the side (like autoregen.sh). It would be nice / useful to be > > able to it without configuring / building anything. For instance, the > > autoregen buildbot job could run it without configuring anything. > > Ideally, the buildbot wouldn't maintain its own autoregen.py file on the > > side, it would just use whatever is in the repo. > > Firstly because of what you mention later: some regeneration steps > require building host tools first, like the XXX-gen in opcodes. > > Since the existing Makefiles already contain the rules to autoregen > all these files, it seemed natural to me to reuse them, to avoid > reinventing the wheel with the risk of introducing new bugs. > > This involves changes in places where I've never looked at before, so > I'd rather reuse as much existing support as possible. > > For instance, there are the generators in opcodes/, but also things in > sim/, bfd/, updates to the docs and potfiles. In gcc, there's also > something "unusual" in fixincludes/ and libgfortran/ > > In fact, I considered also including 'configure', 'Makefile.in', > etc... in the 'regenerate' target, it does not seem natural to me to > invoke a script on the side, where you have to replicate the behaviour > of existing Makefiles, possibly getting out-of-sync when someone > forgets to update either Makefile or autoregen.py. What is currently > missing is a way to easily regenerate files without having to run a > full 'make all' (which currently takes care of calling autoconf & > friends to update configure/Makefile.in). > > But yeah, having to configure before being able to regenerate files is > a bit awkward too :-) > > > > > > Looking at the rule to re-generate copying.c in gdb for instance: > > > > # Make copying.c from COPYING > > $(srcdir)/copying.c: @MAINTAINER_MODE_TRUE@ $(srcdir)/../COPYING3 > > $(srcdir)/copying.awk > >awk -f $(srcdir)/copying.awk \ > >< $(srcdir)/../COPYING3 > $(srcdir)/copying.tmp > >mv $(srcdir)/copying.tmp $(srcdir)/copying.c > > > > There is nothing in this code that requires having configured the source > > tree. This code could for instance be moved to some > > generate-copying-c.sh script. generate-copying-c.sh could be called by > > an hypothetical autoregen.sh script, as well as the copying.c Makefile > > target, if we want to continue supporting the maintainer mode. > Wouldn't it be more obscure than now? Currently such build rules are > all in the relevant Makefile. You'd have to open several scripts to > discover what's involved with updating copying.c > Yeah I agree that it's good to keep all build rules in the Makefile; if there's a possibility of something changing, things that depend up
Re: [RFC] add regenerate Makefile target
> "Eric" == Eric Gallager writes: Eric> Also there are the files generated by cgen, too, which no one seems to Eric> know how to regenerate, either. I thought I sent out some info on this a while ago. Anyway what I do is make a symlink to the cgen source tree in the binutils-gdb source tree, then configure with --enable-cgen-maint. Then I make sure to build with 'make GUILE=guile3.0'. It could be better but that would require someone to actually work on cgen. Eric> And then in bfd there's that chew Eric> program in the doc subdir. And then in the binutils subdirectory Eric> proper there's that sysinfo tool for generating sysroff.[ch]. gdb used to use a mish-mash of different approaches, some quite strange, but over the last few years we standardized on Python scripts that generate files. They're written to be seamless -- just invoke in the source dir; the output is then just part of your patch. No special configure options are needed. On the whole this has been a big improvement. Tom
Re: [PATCH 5/4] libbacktrace: improve getting debug information for loaded dlls
Am 25.01.2024 um 23:04 schrieb Ian Lance Taylor:
On Thu, Jan 25, 2024 at 11:53 AM Björn Schäpers wrote:
Am 23.01.2024 um 23:37 schrieb Ian Lance Taylor:
On Thu, Jan 4, 2024 at 2:33 PM Björn Schäpers wrote:
Am 03.01.2024 um 00:12 schrieb Björn Schäpers:
Am 30.11.2023 um 20:53 schrieb Ian Lance Taylor:
On Fri, Jan 20, 2023 at 2:55 AM Björn Schäpers wrote:
From: Björn Schäpers
Fixes https://github.com/ianlancetaylor/libbacktrace/issues/53, except
that libraries loaded after the backtrace_initialize are not handled.
But as far as I can see that's the same for elf.
Thanks, but I don't want a patch that loops using goto statements.
Please rewrite to avoid that. It may be simpler to call a function.
Also starting with a module count of 1000 seems like a lot. Do
typical Windows programs load that many modules?
Ian
Rewritten using a function.
If that is commited, could you attribute that commit to me (--author="Björn
Schäpers ")?
Thanks and kind regards,
Björn.
I noticed that under 64 bit libraries loaded with LoadLibrary were missing.
EnumProcessModules stated the correct number of modules, but did not fill the
the HMODULEs, but set them to 0. While trying to investigate I noticed if I do
the very same thing from main (in C++) I even got fewer module HMODULEs.
So I went a different way. This detects all libraries correctly, in 32 and 64
bit. The question is, if it should be a patch on top of the previous, or should
they be merged, or even only this solution and drop the EnumProcessModules
variant?
Is there any reason to use both patches? Seems simpler to just use
this one if it works. Thanks.
Ian
This one needs the tlhelp32 header and its functions, which are (accoridng to
the microsoft documentation) are only available since Windows XP rsp. Windows
Server 2003.
If that's no problem, and in my opinion it shouldn't be, then I can basically
drop patch 4 and rebase this one.
I don't see that as a problem. It seems like the patch will fall back
cleanly on ancient Windows and simply fail to pick up other loaded
DLLs. I think that is fine. I'll look for an updated patch. Thanks.
Ian
Attached is the combined version of the two patches, only implementing the
variant with the tlhelp32 API.
Tested on x86 and x86_64 windows.
Kind regards,
Björn.From 33a6380feff66e32ef0f0d7cbad6fb55319f4e2f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Bj=C3=B6rn=20Sch=C3=A4pers?=
Date: Sun, 30 Apr 2023 23:54:32 +0200
Subject: [PATCH 1/2] libbacktrace: get debug information for loaded dlls
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Fixes https://github.com/ianlancetaylor/libbacktrace/issues/53, except
that libraries loaded after the backtrace_initialize are not handled.
But as far as I can see that's the same for elf.
Tested on x86_64-linux and i686-w64-mingw32.
-- >8 --
libbacktrace/Changelog:
* configure.ac: Checked for tlhelp32.h
* configure: Regenerate.
* config.h.in: Regenerate.
* pecoff.c: Include if available.
(backtrace_initialize): Use tlhelp32 api for a snapshot to
detect loaded modules.
(coff_add): New argument for the module handle of the file,
to get the base address.
Signed-off-by: Björn Schäpers
---
libbacktrace/config.h.in | 3 +
libbacktrace/configure| 185 --
libbacktrace/configure.ac | 4 +
libbacktrace/pecoff.c | 74 +--
4 files changed, 171 insertions(+), 95 deletions(-)
diff --git a/libbacktrace/config.h.in b/libbacktrace/config.h.in
index ee2616335c7..9b8ab88ab63 100644
--- a/libbacktrace/config.h.in
+++ b/libbacktrace/config.h.in
@@ -101,6 +101,9 @@
/* Define to 1 if you have the header file. */
#undef HAVE_SYS_TYPES_H
+/* Define to 1 if you have the header file. */
+#undef HAVE_TLHELP32_H
+
/* Define to 1 if you have the header file. */
#undef HAVE_UNISTD_H
diff --git a/libbacktrace/configure b/libbacktrace/configure
index 7ade966b54d..ca52ee3bafb 100755
--- a/libbacktrace/configure
+++ b/libbacktrace/configure
@@ -1866,7 +1866,7 @@ else
#define $2 innocuous_$2
/* System header to define __stub macros and hopefully few prototypes,
-which can conflict with char $2 (); below.
+which can conflict with char $2 (void); below.
Prefer to if __STDC__ is defined, since
exists even on freestanding compilers. */
@@ -1884,7 +1884,7 @@ else
#ifdef __cplusplus
extern "C"
#endif
-char $2 ();
+char $2 (void);
/* The GNU C library defines this for functions which it implements
to always fail with ENOSYS. Some functions are actually named
something starting with __ and the normal name is an alias. */
@@ -1893,7 +1893,7 @@ choke me
#endif
int
-main ()
+main (void)
{
return $2 ();
;
@@ -1932,7 +1932,7 @@ else
/* end confdefs.h. */
$4
int
-main ()
+main (void)
{
if (sizeof ($2))
return 0;
@@ -1
Re: [PATCH 6/4] libbacktrace: Add loaded dlls after initialize
Am 10.01.2024 um 13:34 schrieb Eli Zaretskii:
Date: Tue, 9 Jan 2024 21:02:44 +0100
Cc: i...@google.com, gcc-patc...@gcc.gnu.org, gcc@gcc.gnu.org
From: Björn Schäpers
Am 07.01.2024 um 18:03 schrieb Eli Zaretskii:
In that case, you an call either GetModuleHandeExA or
GetModuleHandeExW, the difference is minor.
Here an updated version without relying on TEXT or TCHAR, directly calling
GetModuleHandleExW.
Thanks, this LGTM (but I couldn't test it, I just looked at the
sour ce code).
Here an updated version. It is rebased on the combined approach of getting the
loaded DLLs and two minor changes to suppress warnings.
From 55d0f312c0a9c4e2305d72fa2329b37937a02e44 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Bj=C3=B6rn=20Sch=C3=A4pers?=
Date: Sat, 6 Jan 2024 22:53:54 +0100
Subject: [PATCH 2/2] libbacktrace: Add loaded dlls after initialize
libbacktrace/Changelog:
* pecoff.c [HAVE_WINDOWS_H]:
(dll_notification_data): Added
(dll_notification_context): Added
(dll_notification): Added
(backtrace_initialize): Use LdrRegisterDllNotification to load
debug information of later loaded
dlls.
---
libbacktrace/pecoff.c | 106 ++
1 file changed, 106 insertions(+)
diff --git a/libbacktrace/pecoff.c b/libbacktrace/pecoff.c
index faa0be0b700..455a5c2191d 100644
--- a/libbacktrace/pecoff.c
+++ b/libbacktrace/pecoff.c
@@ -61,6 +61,34 @@ POSSIBILITY OF SUCH DAMAGE. */
#undef Module32Next
#endif
#endif
+
+#if defined(_ARM_)
+#define NTAPI
+#else
+#define NTAPI __stdcall
+#endif
+
+/* This is a simplified (but binary compatible) version of what Microsoft
+ defines in their documentation. */
+struct dll_notifcation_data
+{
+ ULONG reserved;
+ /* The name as UNICODE_STRING struct. */
+ PVOID full_dll_name;
+ PVOID base_dll_name;
+ PVOID dll_base;
+ ULONG size_of_image;
+};
+
+#define LDR_DLL_NOTIFICATION_REASON_LOADED 1
+
+typedef LONG NTSTATUS;
+typedef VOID CALLBACK (*LDR_DLL_NOTIFICATION)(ULONG,
+ struct dll_notifcation_data*,
+ PVOID);
+typedef NTSTATUS NTAPI (*LDR_REGISTER_FUNCTION)(ULONG,
+ LDR_DLL_NOTIFICATION, PVOID,
+ PVOID*);
#endif
/* Coff file header. */
@@ -911,6 +939,53 @@ coff_add (struct backtrace_state *state, int descriptor,
return 0;
}
+#ifdef HAVE_WINDOWS_H
+struct dll_notification_context
+{
+ struct backtrace_state *state;
+ backtrace_error_callback error_callback;
+ void *data;
+};
+
+static VOID CALLBACK
+dll_notification (ULONG reason,
+ struct dll_notifcation_data *notification_data,
+ PVOID context)
+{
+ char module_name[MAX_PATH];
+ int descriptor;
+ struct dll_notification_context* dll_context =
+(struct dll_notification_context*) context;
+ struct backtrace_state *state = dll_context->state;
+ void *data = dll_context->data;
+ backtrace_error_callback error_callback = dll_context->data;
+ fileline fileline;
+ int found_sym;
+ int found_dwarf;
+ HMODULE module_handle;
+
+ if (reason != LDR_DLL_NOTIFICATION_REASON_LOADED)
+return;
+
+ if (!GetModuleHandleExW (GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS
+ | GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT,
+ (wchar_t*) notification_data->dll_base,
+ &module_handle))
+return;
+
+ if (!GetModuleFileNameA ((HMODULE) module_handle, module_name, MAX_PATH - 1))
+return;
+
+ descriptor = backtrace_open (module_name, error_callback, data, NULL);
+
+ if (descriptor < 0)
+return;
+
+ coff_add (state, descriptor, error_callback, data, &fileline, &found_sym,
+ &found_dwarf, (uintptr_t) module_handle);
+}
+#endif
+
/* Initialize the backtrace data we need from an ELF executable. At
the ELF level, all we need to do is find the debug info
sections. */
@@ -935,6 +1010,8 @@ backtrace_initialize (struct backtrace_state *state,
#endif
#ifdef HAVE_WINDOWS_H
+ HMODULE nt_dll_handle;
+
module_handle = (uintptr_t) GetModuleHandle (NULL);
#endif
@@ -981,6 +1058,35 @@ backtrace_initialize (struct backtrace_state *state,
}
#endif
+#ifdef HAVE_WINDOWS_H
+ nt_dll_handle = GetModuleHandleW (L"ntdll.dll");
+ if (nt_dll_handle)
+{
+ LDR_REGISTER_FUNCTION register_func;
+ const char register_name[] = "LdrRegisterDllNotification";
+ register_func = (void*) GetProcAddress (nt_dll_handle,
+ register_name);
+
+ if (register_func)
+ {
+ PVOID cookie;
+ struct dll_notification_context *context
+ = backtrace_alloc (state,
+ sizeof(struct dll_notification_context),
+ error_callback, data);
+
+ if
gcc-12-20240315 is now available
Snapshot gcc-12-20240315 is now available on https://gcc.gnu.org/pub/gcc/snapshots/12-20240315/ and on various mirrors, see https://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 12 git branch with the following options: git://gcc.gnu.org/git/gcc.git branch releases/gcc-12 revision caabffc463db169e4a54deade79abe0888df05b5 You'll find: gcc-12-20240315.tar.xz Complete GCC SHA256=23f757c6ba564f3b2fe561b0539d8396e264dc9af3876c08c7cf63b1aaa93e62 SHA1=8e104f0952446a241568bbc6f80d75e89c3435b9 Diffs from 12-20240308 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-12 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
