Re: ASAN test failures make compare_tests useless
Not sure I understand what the problem is. Responded inline. On Mon, Aug 18, 2014 at 9:43 AM, Yury Gribov wrote: > On 08/18/2014 09:42 AM, Yury Gribov wrote: >> >> On 08/16/2014 04:37 AM, Manuel López-Ibáñez wrote: >>> >>> On the compile farm, ASAN tests seem to fail a lot like: >>> >>> FAIL: c-c++-common/asan/global-overflow-1.c -O0 output pattern >>> test, is ==31166==ERROR: AddressSanitizer failed to allocate >>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >>> 12) >>> ==31166==ReserveShadowMemoryRange failed while trying to map >>> 0xdfff0001000 bytes. Perhaps you're using ulimit -v >>> , should match READ of size 1 at 0x[0-9a-f]+ thread T0.*( Sounds like the tests do not even start up properly. No mmap failures should be reported. >>> The problem is that those addresses and sizes are very random, The output pattern that must be printed has these addresses masked out (note "0x[0-9a-f]+" in your report). No other lines with varying addresses should be printed. >>> so when >>> I compare the test results of a pristine trunk with a patched one, I >>> get: >>> >>> New tests that FAIL: >>> >>> unix//-m64: c-c++-common/asan/global-overflow-1.c -O0 output >>> pattern test, is ==12875==ERROR: AddressSanitizer failed to allocate >>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >>> 12) >>> unix//-m64: c-c++-common/asan/global-overflow-1.c -O0 output >>> pattern test, is ==18428==ERROR: AddressSanitizer failed to allocate >>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >>> 12) >>> [... hundreds of ASAN tests that failed...] >>> >>> Old tests that failed, that have disappeared: (Eeek!) >>> >>> unix//-m64: c-c++-common/asan/global-overflow-1.c -O0 output >>> pattern test, is ==30142==ERROR: AddressSanitizer failed to allocate >>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >>> 12) >>> unix//-m64: c-c++-common/asan/global-overflow-1.c -O0 output >>> pattern test, is ==31166==ERROR: AddressSanitizer failed to allocate >>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >>> 12) >>> [... the same hundreds of tests that already failed before...] >>> >>> The above makes very difficult to identify failures caused by my patch. >>> >>> Can we remove the "==" part of the error? This way compare_tests >>> will ignore the failures. Am I understanding correctly that "==" in the test stdout has some special meaning for compare_tests (whatever they are, I'm not really familiar with GCC testing infrastructure)? If so, this is quite a questionable choice (e.g. Valgrind also prefixes the report lines with "==12345=="), and I don't see the point in removing PIDs/addresses to please this script. >>> Alternatively, I could patch compare_tests to sed out that part before >>> comparing. Would that be acceptable? >>> >>> Cheers, >>> >>> Manuel. >>> >> >> Added Sanitizer folks. Frankly it'd be cool if dumping PIDs and >> addresses could be turned off. >> > > Ok, this time actually added them. > -- Alexander Potapenko Software Engineer Google Moscow
Re: ASAN test failures make compare_tests useless
On Mon, Aug 18, 2014 at 9:42 AM, Yury Gribov wrote: > On 08/16/2014 04:37 AM, Manuel López-Ibáñez wrote: >> >> On the compile farm, ASAN tests seem to fail a lot like: >> >> FAIL: c-c++-common/asan/global-overflow-1.c -O0 output pattern >> test, is ==31166==ERROR: AddressSanitizer failed to allocate >> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >> 12) >> ==31166==ReserveShadowMemoryRange failed while trying to map >> 0xdfff0001000 bytes. Perhaps you're using ulimit -v >> , should match READ of size 1 at 0x[0-9a-f]+ thread T0.*( >> >> The problem is that those addresses and sizes are very random, so when >> I compare the test results of a pristine trunk with a patched one, I >> get: >> >> New tests that FAIL: >> >> unix//-m64: c-c++-common/asan/global-overflow-1.c -O0 output >> pattern test, is ==12875==ERROR: AddressSanitizer failed to allocate >> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >> 12) >> unix//-m64: c-c++-common/asan/global-overflow-1.c -O0 output >> pattern test, is ==18428==ERROR: AddressSanitizer failed to allocate >> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >> 12) >> [... hundreds of ASAN tests that failed...] >> >> Old tests that failed, that have disappeared: (Eeek!) >> >> unix//-m64: c-c++-common/asan/global-overflow-1.c -O0 output >> pattern test, is ==30142==ERROR: AddressSanitizer failed to allocate >> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >> 12) >> unix//-m64: c-c++-common/asan/global-overflow-1.c -O0 output >> pattern test, is ==31166==ERROR: AddressSanitizer failed to allocate >> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno: >> 12) >> [... the same hundreds of tests that already failed before...] >> >> The above makes very difficult to identify failures caused by my patch. >> >> Can we remove the "==" part of the error? This way compare_tests >> will ignore the failures. >> >> Alternatively, I could patch compare_tests to sed out that part before >> comparing. Would that be acceptable? >> >> Cheers, >> >> Manuel. >> > > Added Sanitizer folks. Frankly it'd be cool if dumping PIDs and addresses > could be turned off. > Could you please name a reason for that? Doing so complicates the debugging of multi-process applications but doesn't bring any obvious advantages. -- Alexander Potapenko Software Engineer Google Moscow
Problem with static constructor priorities on Darwin
Hi Geoff, Richard et al., Looks like there's a problem with the support for static constructor priorities when targeting Darwin. This manifests under AddressSanitizer (gcc/asan.c), which inserts an additional static constructor with priority equal to MAX_RESERVED_INIT_PRIORITY-1 per each module - however the constructor ends up being the last one in the __mod_init_func section, which leads to crashes (see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55617) Looking at gcc/config/darwin.c I suspect that this is because machopic_asm_out_constructor() just emits a pointer into __mod_init_func for each constructor irrespective of its priority. This might be fine if the constructors are created by a single compiler pass (at least non-ASan builds behave correctly - maybe someone else takes care of the priorities), but everything stops working if another pass tries to add its own constructor. As a proof of concept I've modified gcc/config/darwin.c to save the constructors in machopic_asm_out_constructor() and emit them in reversed order in darwin_file_end() (see http://gcc.gnu.org/bugzilla/attachment.cgi?id=29309&action=diff) - this really changes the order of constructors and makes small ASan tests pass, but this solution is far from a complete one. Is my understanding of the current problem with machopic_asm_out_constructor() correct? If so, any chance it can be fixed? Thanks, Alexander Potapenko Software Engineer Google Moscow
Re: About BZ#87210 [RFE] To initialize automatic stack variables
On Mon, Mar 4, 2019 at 9:46 PM David Brown wrote:
>
> On 19/02/2019 11:23, P J P wrote:
> > Hello,
> >-> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87210
> >
> > This RFE is about providing gcc option(s) to eliminate information leakage
> > issues from programs. Information leakage via uninitialised memory has
> > beena chronic/recurring issue across all software. They are found quite
> > often andmay lead to severe effects if found in system software/kernel, OR
> > an applicationwhich handles sensitive information.
> > Various projects/efforts are underway to keep such information exposurefrom
> > happening
> > * STACKLEAK - http://lkml.iu.edu/hypermail/linux/kernel/1810.3/00522.html
> > * KLEAK - https://netbsd.org/gallery/presentations/maxv/kleak.pdf*
> > https://j00ru.vexillium.org/papers/2018/bochspwn_reloaded.pdf
> > But these are still external corrections to improve specific project
> > and/orsoftware. It does not help to fix/eliminate all information leakage
> > issues.
> > Automatic memory initialisation:
> >
> > * https://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html
> > * https://reviews.llvm.org/D54604
> > It'd be immensely helpful and welcome if gcc(1) could provide
> > compile/buildtime options to enable/disable - automatic memory
> > initialisation.
> > Could we please consider it as more viable/useful option?
> > Thank you.---
> >-P J P
> > http://feedmug.com
> >
>
> This strikes me as getting the issue completely backwards.
>
> It is not lack of initialisation of stack variables that leads to
> information leakage - it is a failure to clear the important data left
> behind on the stack.
Well, yes and no.
If we consider only a handful of variables containing important data,
we can probably afford to annotate them to be cleared upon leaving the
function using an attribute you're suggesting below.
(Although humans are really bad at annotating code - such attributes
go out of sync in no time).
But e.g. in the Linux kernel even a leak of a single word or a pointer
can help malicious users to defeat ASLR, stack canaries etc.
Now, if we choose to clear _every_ local variable when we leave the
function, we'll have to memset the whole stack frame, which is costly
(as you mention, we don't want the memset to be optimized away).
At the same time, memsetting locals upon function entry gives us the
same result, but has a bigger optimization potential (most of the time
one won't have to pre-initialize anything, thanks to DSE).
> The problem with information leakage is when you have something like:
>
> void foo(void) {
> char key[20];
> strcpy(key, "Top secret!");
> usekey(&key);
> memset(key, 0, sizeof(key));// optimised away by DSE
> }
>
> void bar(void) {
> foo();
> char stolen_key[40];// Covering "key" and other stack
> steal_key(&stolen_key);
> }
>
>
> Forcing "stolen_key" to be zero initialised does not help anyone -
> options for that just make code slower and hide errors that would occur
> with other compiler options. The challenge is to make sure /key/ is
> zeroed out after use - no matter what optimisations, and whether or not
> the "memset" is called.
>
> gcc already has mechanisms for handling this.
>
> First, there is a way to tell gcc that something in memory will be read,
> even though it doesn't look like it:
>
> void foo(void) {
> char key[20];
> strcpy(key, "Top secret");
> usekey(key);
> memset(key, 0, sizeof(key));
> {
> typedef struct { char x[20]; } XS;
> XS *p = (XS *) key;
> asm("" : "+m" (*p));
> }
> }
>
> Next, to automate the clearing of the key regardless of how and when the
> function "foo" is exited and whether or not "memset" is called, we can
> use the "cleanup" attribute:
>
> static void clearKey(char (*key)[20]) {
> memset(key, 0, 20);
> {
> typedef struct { char x[20]; } XS;
> XS *p = (XS *) key;
> asm("" : "+m" (*p));
> }
> }
>
> void foo2(void) {
> char key[20] __attribute__((cleanup(clearKey)));
> strcpy(key, "Top secret");
> usekey(key);
> }
>
> This stops information leakage where it should be stopped - once the
> information is no longer used. Forcing initialisation of stack
> variables would put it in the wrong place, when the stack space is reused.
>
> And the code generated here is as good as i
