Re: compile time regression
This is now in bugzilla as: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=28871 -benjamin
Re: regress and -m64
When I run bootstrap and "make check", I check the -m64 option (only). Check gcc-testresults. Currently, the results don't look very good. Maybe I'm doing something wrong. Brad
Re: regress and -m64
Brad,
Why don't you try a normal multi-lib build without any
extra flags. At the moment, considering how much noise is on
the testsuite results on Darwin due to this linker warnings, I
don't think its really helpful to bother exploring corner cases
of building gcc trunk with unique flags. Make sure you have...
--- gcc-4.2-20060825/libstdc++-v3/testsuite/lib/prune.exp.org 2006-08-26
11:22:52.0 -0400
+++ gcc-4.2-20060825/libstdc++-v3/testsuite/lib/prune.exp 2006-08-26
11:23:39.0 -0400
@@ -29,5 +29,7 @@
regsub -all "(^|\n)\[^\n\]*: Additional NOP may be necessary to workaround
Itanium processor A/B step errata" $text "" text
regsub -all "(^|\n)\[^\n*\]*: Assembler messages:\[^\n\]*" $text "" text
+regsub -all "(^|\n)can't find atom for N_GSYM stabs \[^\n\]* in \[^\n\]*"
$text "" text
+
return $text
}
--- gcc-4.2-20060822/gcc/testsuite/lib/prune.exp.org2006-08-23
18:33:56.0 -0400
+++ gcc-4.2-20060822/gcc/testsuite/lib/prune.exp2006-08-23
18:41:28.0 -0400
@@ -43,6 +43,7 @@
regsub -all "(^|\n)\[^\n\]*file path prefix \[^\n\]* never used" $text ""
text
regsub -all "(^|\n)\[^\n\]*linker input file unused since linking not
done" $text "" text
+regsub -all "(^|\n)can't find atom for N_GSYM stabs \[^\n\]* in \[^\n\]*"
$text "" text
#send_user "After:$text\n"
return $text
...applied. You will also need to apply Geoff's proposed TImode patch
from http://gcc.gnu.org/ml/gcc-patches/2006-08/msg00581.html. Without
the TImode patch, libgfortran at least will be horribly broken since
it relies on libgcc being able to provide the TImode symbols for its
long long math operations. This will eliminate about 800 failures
from the gfortran testsuite. You need the patches above for ignoring
the linker warnings from ld64 to eliminate the remaining 40 some
false failures from the gfortran testsuite. The same is true for
libstdc++ which only really has 4 additional failures under -m64.
Jack
ps I have only patched the prune.exp to fix the builds of the
c,c++ and fortran languages. If you see massive failures for
any other language's testsuite, just look in the log for errors
of the form...
can't find atom for N_GSYM stabs
...if you seem them, the prune.exp being used for that
section of the testsuite needs a patch like those above.
Oh, when you do your make check from the top level of the
build directory use this form of the command...
make -k check RUNTESTFLAGS='--target_board="unix{,-m64}"'
Re: First cut on outputing gimple for LTO using DWARF3. Discussion invited!!!!
On Aug 28, 2006, at 6:57 AM, Kenneth Zadeck wrote: This posting is a progress report of my task of encoding and decoding the GIMPLE stream into LTO. Included in this posting is a patch that encodes functions and dumps the result to files. Interesting email. One question: how big are the files you're generating? For example, how much space does it take to store the body of a function with 100 add operations in it (before gzip'ing it)? What is the size proportional to? # instructions, # basic blocks, what else? -Chris
Re: regress and -m64
On Aug 28, 2006, at 12:10 PM, Jack Howarth wrote:
Why don't you try a normal multi-lib build without any
extra flags.
What extra flags? The configure command is
../configure --prefix=/pkgs/gcc-mainline --with-gmp=/opt/local/ --
with-mpfr=/opt/local/
which is totally generic (one needs to specify gmp and mpfr libraries
to get fortran to build).
I build with
make -j 4 bootstrap >& build.log
which is completely generic.
Or do you mean the -mcpu=970 in the test options?
Oh, when you do your make check from the top level of the
build directory use this form of the command...
make -k check RUNTESTFLAGS='--target_board="unix{,-m64}"'
my make check is
make -k -j 8 check RUNTESTFLAGS="--target_board 'unix{-mcpu=970/-m64}'"
I don't see any reason to check the 32-bit stuff that the regression
checker checks at least once a day.
Brad
Re: First cut on outputing gimple for LTO using DWARF3. Discussion invited!!!!
Chris Lattner wrote: > On Aug 28, 2006, at 6:57 AM, Kenneth Zadeck wrote: > >> This posting is a progress report of my task of encoding and decoding >> the GIMPLE stream into LTO. Included in this posting is a patch that >> encodes functions and dumps the result to files. > > Interesting email. One question: how big are the files you're > generating? For example, how much space does it take to store the > body of a function with 100 add operations in it (before gzip'ing > it)? What is the size proportional to? # instructions, # basic > blocks, what else? > > -Chris The size will be large because the trees are still large. Encoding everything in a bootstrap for the three languages was on the order of 60mb. As you know, size is a well known problem in gcc. The plan is to track the changes in gimple and as that gets smaller, this will get smaller. I would like to get the types and perhaps the flags out of the internal nodes of the trees. when that gets done, these operations should be small. I actually do not think that it is productive to spend that much time measuring this until we first assure ourselves that we are getting all of the information output correctly. That 60mb number will change a lot (both up and down) as we get closer to getting everything tied together. Then we can spend more time looking at the pieces and seeing where we need to do better.
Re: Potential fix for rdar://4658012
Richard Kenner wrote: > I disagree. Testing is not, and should never be, a substitute for analysis. > > A patch is proposed because we have a reason to believe it's correct. Then > we test to increase our confidence that it is, indeed, correct. But both > parts are essential for any patch. > > Here we have a situation where we'd like to see some optimization done (the > merging of temporaries). There's some code that suppresses that optimization > in one case because that optimization was unsafe in that case. It is not > acceptable to find out whether that code is still needed by merely running > a test suite: there has to be some analysis that says what changed to > make that code no longer needed. The burden of proof on that analysis has > to be on the person who's proposing that the code is no longer needed. Richard - I agree with you -- my intent for originally posting was to hopefully find that understanding of why the code had originally been added. Perhaps I should have better explained my own understanding, so that you could help identify if this was a conceptual mistake on my part. I think I had too much confidence that this would be an easy question to answer, but it appears that the code is quite dated and not well understood. So, without further ado, let me provide my analysis, and hopefully those who understand this code best can help point out if I've gone astray. Here is my interpretation of this code: In the code for managing middle-end-generated temps, it appears that push/pop_temp_slots is used to create nested contexts into which unique temps are generated. When I need a temp, I call push_temp_slots, allocate a temp, perform my calculation, and call pop_temp_slots when I'm done. The one exception to this is if the address of the temp is taken before I call pop_temp_slots. In that instance, even though I may be "done" with the temp, it needs to live until the end of the high-level-language scope, and so it is marked with addr_taken and is pushed up in the temp scopes using preserve_temp_slots. It is this logic that is used for function calls that return a value on the stack. However, in the case where we're passing the address of a temp slot to a function, it doesn't make sense to me that this is the same as other "address-of" operations on a stack slot. The function call (at least in C and C++) cannot preserve this address, and it is reasonable to say that attempts to access this address after the caller is done with the location, are invalid. So, for these reasons, marking the temp for a lifetime longer than what is needed by the middle-end for copying it into the destination, appeared to be excessive. Hopefully this is enough of an explanation to reveal whether my understanding is consistent or inconsistent with the experts, and can move the discussion forward. Thanks again for all of the time looking at this - it's very much appreciated. - Josh
Re: regress and -m64
Brad, I was confusing your use of -mcpu=970 in the make check with the build flags...sorry. You might stop using that flag for now until you get a baseline of how many additional failures seen in -m64 compared to -m32 are not due to the linker warnings (after you apply the TImode patch). I'll look at my last make check when I get home tonight but I don't think you should see more than an additional 20 failures for g++ and an additional 4 failures for libstdc++. The bulk of the failures are seeing must be either due to 1) the use of -mcpu=970, 2) the absence of the TImode patch or 2) not pruning the linker warnings. Jack ps I would first try retesting your current build without -mcpu=970 and then rebuild with the TImode patch and test again without -mcpu=970. Then you can patch the prune.exp and test a third time. This should give you how many failures are due to each cause.
Re: regress and -m64
On Aug 27, 2006, at 7:24 AM, Jack Howarth wrote: Can one of you remind me who we need to lobby at Apple In the gcc project, contributions are generally speaking, made by an individual. Geoff operates a regression tester, probably the one you're thinking of. In the past, he has considered building/testing non-default things, and has generally shied away from non-default things. Bear in mind, the addition of -m64 comes at the cost of being able to test less often. He isn't the only one known to contribute test results on darwin however. I'd certainly welcome a contribution of testing that included -m64, if someone has the time, resources and motivation.
Re: regress and -m64
Mike, Do you know if regress is used for anything other than building and checking gcc? Also is it a dual G5 by any chance? I ask because it is unclear if regress is doing a 'make -k -j 2 check' or not? Might that not increase the rate of testing on regress? I haven't tried that myself because I tend to use my machine during builds and make checks. However if regress is just sitting there doing gcc builds/checks, it might be worth trying that. It appears to be completeing a build/check cycle every 30 hours. Perhaps that could be significantly cut back if both processors were used. Jack
Re: First cut on outputing gimple for LTO using DWARF3. Discussion invited!!!!
On Aug 28, 2006, at 10:36 AM, Kenneth Zadeck wrote: I actually do not think that it is productive to spend that much time measuring this until we first assure ourselves that we are getting all of the information output correctly. That 60mb number will change a lot (both up and down) as we get closer to getting everything tied together. Then we can spend more time looking at the pieces and seeing where we need to do better. Okay, thanks! -Chris
Re: regress and -m64
On Aug 28, 2006, at 1:20 PM, Jack Howarth wrote: Might that not increase the rate of testing on regress? Sorry, nope.
Re: gcc trunk vs python
On 8/27/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: ... I have not received reports about bugs in the offending code when compiled with other compilers. I do know at least one another compiler that does this, and at least one significant project (which is actually quite a bit larger than Python) that suffered similar problem as Python's suffering - in short, this isn't new and this isn't even something gcc specific (or fault). And Python isn't the first program that suffers from this either. > code. But this particular optimization will let us do a much better > job on very simple loops like > for (int i = 0; i != n; i += 5) > The issue with a loop like this is that if signed overflow is defined > to be like unsigned overflow, then this loop could plausibly wrap > around INT_MAX and then terminate. If signed overflow is undefined, > then we do not have to worry about and correctly handle that case. That seems to me rather obviously broken code unless the programmer has proven to himself that n is a multiple of 5. So why bother attempting to optimize it? There are legitimate code with similar form, like pointer increment compared against the "end" pointer which happens often in string manipulation or buffer management code. More importantly some of C++ STL iterators often end up in such a form. -- #pragma ident "Seongbae Park, compiler, http://seongbae.blogspot.com";
Re: regress and -m64
Mike, Well then alternatively, might not 'make -j 2' increase the rate at which gcc is built on regress? Or doesn't Darwin support parallel builds? If we can't speed up the testing then we could at least speed up the build process to free up time for -m64. After all, we are already building -m64 so it is more important to actually test what we build than to add new languages to the build process on regress. Jack
Re: regress and -m64
On Aug 28, 2006, at 3:32 PM, Jack Howarth wrote: Well then alternatively, might not 'make -j 2' increase the rate at which gcc is built on regress? Yes, we know about -j2. When I said, sorry, nope, I meant to convey the idea that in fact that adding a -j2 won't speed it up. Or doesn't Darwin support parallel builds? Yes, of course it does. If we can't speed up the testing ? -j2 makes testing go faster as well. then we could at least speed up the build process to free up time for -m64.
Re: regress and -m64
Mike, Now I totally confused. Are you saying that '-j 2' only speeds up testing but not builds? In one sentence you say it won't speed things up and in the next you say of course it speeds things up. Which is it? Jack On Aug 28, 2006, at 3:32 PM, Jack Howarth wrote: > Well then alternatively, might not 'make -j 2' increase the rate at > which gcc is built on regress? Yes, we know about -j2. When I said, sorry, nope, I meant to convey the idea that in fact that adding a -j2 won't speed it up. > Or doesn't Darwin support parallel builds? Yes, of course it does. > If we can't speed up the testing ? -j2 makes testing go faster as well. > then we could at least speed up the build process to free up time > for -m64.
Re: regress and -m64
On Aug 28, 2006, at 3:57 PM, Jack Howarth wrote: > If we can't speed up the testing ? -j2 makes testing go faster as well. Sigh, I misstated that one. My comment in that case was about the general case. I meant to say that -j2 is as applicable to testing as it is to building. Let me try once more, yes, we already know about adding -j2, no, it won't help.
Re: regress and -m64
Mike, Okay. How about this. Have regress set to at least do a -m64 make check once a week. At least that would show some interest in the status of gcc at 64-bit. It is baffling to most people how we can make any progress on such issues if the status isn't sampled at least once every blue moon. Jack
Re: regress and -m64
On Aug 28, 2006, at 5:08 PM, Jack Howarth wrote: Okay. How about this. Have regress set to at least do a -m64 make check once a week. I think it is a G4, so testing G5 code-gen might have to wait until the G5 emulator is done. :-) Can you contribute G5 results once a week? At least that would show some interest in the status of gcc at 64- bit. It is baffling to most people how we can make any progress on such issues if the status isn't sampled at least once every blue moon. Sampling doesn't actually progress things much, actually contributing code to fix it is what advances it. Bear in mind, by spending all my time talking about it, I have even less time to try and make that happen, which is already hard enough to do.
Re: First cut on outputing gimple for LTO using DWARF3. Discussion invited!!!!
Kenneth Zadeck wrote: This posting is a progress report of my task of encoding and decoding the GIMPLE stream into LTO. Included in this posting is a patch that encodes functions and dumps the result to files. I have only a limited understanding of GIMPLE and LTO, but here are my comments regarding DWARF. DWARF is a debugging format used to describe a program's source and how it is represented in an executable file. GIMPLE is an intermediate representation, not program source. It's always interesting to see one tool used in a different context, but I'm not clear that in this case you are using the right tool for the right task in the way that it was designed. Speaking on behalf of the DWARF Standards Workgroup, we welcome suggestions for extensions or improvements in the DWARF standard. I wasn't able to identify any specific extension in your email (although I didn't look at the patch in detail). If there are specific areas where DWARF can be improved, please let me know. 1) ABBREV TABLES ARE BAD FOR LTO. The abbrev table is a mechanism for making a DWARF3 file self descriptive. It contains a set of templates that are used to describe the records and attributes that are written to describe the object module. This self description mechanism is much weaker (an therefor much more compact) than an XML file's schemata because it depends on a common understanding of the meaning of the tags and attributes that describe the pieces of the data. The tags them selves can be thought of as characters, and an abbrev entry is a magic cookie that tells you how to break the characters into words. DWARF is not self-descriptive, except in the very limited sense that several sections contain version numbers. The DWARF abbreviation table is not intended to be self-descriptive or to be descriptive of the DWARF data. It is a simple compression scheme which compacts the Tag/Attribute/Value structure of a DWARF DIE into a much more condensed format. Certainly the DWARF Abbrev table makes no pretense of being anything similar to an XML schema. I wouldn't use an XML schema for debugging information (it would be huge and slow), and I wouldn't expect to use DWARF abbreviation tables to describe anything at all. However, this mechanism is only self descriptive if you do not extend the set of tags. That is not an option for LTO. While the DWARF3 designers seem to have added every conceivable tag to describe object code, it is not be surprising that this comes up short when describing GIMPLE. While some of GIMPLE could be shoe-horned into the existing tags, all of it will not fit, so anyone who wants to understand GIMPLE, will need to use the GCC headers. The DWARF standard provides for a consistent means to extend the debugging format. If you extend DWARF beyond the standard, as has often been done, you will need to provide descriptions (in some form) of what these extensions mean. You can provide this as written material, or you can have the implementer read the code. In either case, the Abbrev tables only provide compression for the Tag/Attr/Val structure of a DIE. They do no provide any semantic description. Using the abbrev tables also add a lot of complexity. It should be understood that while the abbrev table for two separate files encodes the same information, the way that the abbrev tables are generated almost guarantees that the abbrev table will be encoded differently for each file. The abbrev table is a sequential list of record descriptors. The way that the list is generated is first needed, first output, and you only output the records you need. Each of these records is numbered sequentially. So the numbering of the abbrev records will generally be different for the two created object files since they reference different GIMPLE nodes. There is no need for Abbrev tables to be generated differently for each object file. Several compilers generate a standard Abbrev table and then generate DWARF using these predefined abbreviations. The DWARF standard also allows the Abbrev table to be stored in a shared object and referenced by individual object files. I'm not aware of any actual implementation of this, but the structure is there. I strongly believe that for LTO to work, we are going to have to implement some mechanism where the function bodies are loaded into the compiler on demand (possibly kept in cache, but maybe not). This will be more cumbersome if we have to keep reloading each object file's abbrev table just to tear apart a single function in that .o file. While the abbrev sections average slightly less than %2 of the of the size of the GIMPLE encoding for an entire file, each abbrev table averages about the same size as a single function. Thus, we will either have to keep all the abbrev entries for an entire compilation, implement a second caching mechanism, or do about twice the file reading to load a single function. I'm not exactly sure what you are trying to do
Re: regress and -m64
Mike, Sure. I am actually doing a build right now to demonstrate the baseline we have for c, c++, and fortran with just the TImode patch and the prune.exp patches I use. Hmm, regress is a G4...surely you can find a G5 laying around Apple to replace it with since PowerPC is so passe there (grin). Jack ps I'll start a make check for both -m32/-m64 and post it as soon as its done.
Re: First cut on outputing gimple for LTO using DWARF3. Discussion invited!!!!
Michael Eager wrote: > Kenneth Zadeck wrote: >> This posting is a progress report of my task of encoding and decoding >> the GIMPLE stream into LTO. Included in this posting is a patch that >> encodes functions and dumps the result to files. > > I have only a limited understanding of GIMPLE and LTO, but here are my > comments regarding DWARF. > > DWARF is a debugging format used to describe a program's source and how > it is represented in an executable file. GIMPLE is an intermediate > representation, not program source. It's always interesting to see one > tool used in a different context, but I'm not clear that in this case > you are using the right tool for the right task in the way that it was > designed. > > Speaking on behalf of the DWARF Standards Workgroup, we welcome > suggestions for extensions or improvements in the DWARF standard. > I wasn't able to identify any specific extension in your email (although > I didn't look at the patch in detail). If there are specific areas > where DWARF can be improved, please let me know. > Michael, My original posting was primarily intended for the gcc developer community which is familiar with gimple and not so familiar with dwarf. I actually think that dwarf is probably pretty good for what it was designed for and it is also documented much better than most of the internals of gcc. The task I was given was to encode gimple into dwarf, something that looked like it might be a reasonable idea, but is, in fact, a bad idea. My posting was to tell the gcc community, been there, done that, now lets move on. I will respond to many of your points but you should not take this as criticism of dwarf, these are mostly just places where the interface between gimple and dwarf was particularly troublesome and unless you want to get in the transporting intermediate code business, which I strongly suggest you avoid, you can just ignore most of these problems. > >> 1) ABBREV TABLES ARE BAD FOR LTO. >> >> The abbrev table is a mechanism for making a DWARF3 file self >> descriptive. It contains a set of templates that are used to describe >> the records and attributes that are written to describe the object >> module. >> >> This self description mechanism is much weaker (an therefor much more >> compact) than an XML file's schemata because it depends on a common >> understanding of the meaning of the tags and attributes that describe >> the pieces of the data. The tags them selves can be thought of as >> characters, and an abbrev entry is a magic cookie that tells you how >> to break the characters into words. > > DWARF is not self-descriptive, except in the very limited sense that > several sections contain version numbers. The DWARF abbreviation table > is not intended to be self-descriptive or to be descriptive of the > DWARF data. It is a simple compression scheme which compacts the > Tag/Attribute/Value structure of a DWARF DIE into a much more condensed > format. > > Certainly the DWARF Abbrev table makes no pretense of being anything > similar to an XML schema. I wouldn't use an XML schema for debugging > information (it would be huge and slow), and I wouldn't expect to use > DWARF abbreviation tables to describe anything at all. > While I understand your point, from a language theoretic point of view, they really can be thought of a schemata. They are a description of the records types that have been written by the compiler (or some other tool) for the tags and attributes in this module. This may not be the way that they evolved in your community, but for someone coming from the outside with no knowledge of your history, this is what they are. >> However, this mechanism is only self descriptive if you do not extend >> the set of tags. That is not an option for LTO. While the DWARF3 >> designers seem to have added every conceivable tag to describe object >> code, it is not be surprising that this comes up short when describing >> GIMPLE. While some of GIMPLE could be shoe-horned into the existing >> tags, all of it will not fit, so anyone who wants to understand >> GIMPLE, will need to use the GCC headers. > > The DWARF standard provides for a consistent means to extend > the debugging format. If you extend DWARF beyond the standard, as has > often been done, you will need to provide descriptions (in some form) > of what these extensions mean. You can provide this as written > material, or you can have the implementer read the code. In either > case, the Abbrev tables only provide compression for the Tag/Attr/Val > structure of a DIE. They do no provide any semantic description. > What I did was add 4 new DW_FORMS. This is one of the areas that, I assume, users are not supposed to extend. My assumption is based on the fact that the standard says how to extend tags and attributes, but is silent on how to extend forms. The four forms that I added are: 1) counted list of udata. 2) counted list of sdata. 3) flag with false value. 4) flag w
