Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 00:02, Joseph Myers wrote: > > On Wed, 18 Dec 2019, Joseph Myers wrote: > > > On Mon, 18 Nov 2019, Richard Earnshaw (lists) wrote: > > > > > I've attached a sample from the start of the fixed list - the full list > > > is far > > > too big to post to give a flavour of how the script currently works. Note > > > that annotations of the form [checkme: ] in the summary are for > > > diagnostic > > > purposes. These are where heuristics suggest that there's a higher than > > > normal chance that the PR number is incorrect and that manual auditing is > > > recommended. Such annotations would not be appropriate in the final > > > conversion. > > > > Concretely, here is the current list of 664 checkme: annotations where > > something was suspicious about the PR number (either component mismatch or > > resolved as INVALID). Would some people like to volunteer to pick up > > sections of this list and, for their section, produce a list of SVN > > revisions (at the end of the checkme line) for which the PR number appears > > to be correct, and a list of mappings from SVN revision to correct PR > > number when the PR number appears to be wrong? For any that don't get > > reviewed like that we can easily make the script, for the final > > conversion, decline to add a new summary line for any commit where the PR > > number is suspicious. > > Here's a slightly shorter version with 644 checkme: annotations, after > adding a few more component aliases to the script (e.g., no longer > considering it suspicious if the log message says PR g++/something and > that PR is in the component that's actually called c++). Line 18: c++ SVN r116634, looks suspicious, but PR number is correct. Line 326: lto SVN r196613, PR number is correct Line 411: libstdc++ SVN r219147, PR number is correct How do you want the mapping from SVN revision to correct PR to be expressed? Line 19: the correct PR for fortran SVN r120056 is fortran/30238 (not 39238) Line 608: lto SVN r268728 should be PR 87089 (not 87809) Line 616: lto SVN r269799 should be PR 87089 (not 87809)
Re: can not found mcp/mpfr/gmp, but i has installed them.
On Sat, 14 Dec 2019 at 07:19, 刘 加权 wrote: > > cmd 1 : ./configure --disable-multilib --prefix=/usr/local > --with-mpc=/usr/local --with-mpfr=/usr/local --with-gmp=/usr/local && make > -j4 && sudo make install > cmd 2 : ./configure --disable-multilib --prefix=/usr/local && make -j4 > && sudo make install > > after I installed mpfr mpc gmp , I use --with-mpc=/usr/local to set the > path, > but configure give me the error blew: This is the wrong mailing list for your question, you should have sent it to gcc-help instead. I suggest you follow the advice of https://gcc.gnu.org/wiki/InstallingGCC as it makes things much easier.
Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 09:27, Jonathan Wakely wrote: > > On Thu, 19 Dec 2019 at 00:02, Joseph Myers wrote: > > > > On Wed, 18 Dec 2019, Joseph Myers wrote: > > > > > On Mon, 18 Nov 2019, Richard Earnshaw (lists) wrote: > > > > > > > I've attached a sample from the start of the fixed list - the full list > > > > is far > > > > too big to post to give a flavour of how the script currently works. > > > > Note > > > > that annotations of the form [checkme: ] in the summary are for > > > > diagnostic > > > > purposes. These are where heuristics suggest that there's a higher than > > > > normal chance that the PR number is incorrect and that manual auditing > > > > is > > > > recommended. Such annotations would not be appropriate in the final > > > > conversion. > > > > > > Concretely, here is the current list of 664 checkme: annotations where > > > something was suspicious about the PR number (either component mismatch or > > > resolved as INVALID). Would some people like to volunteer to pick up > > > sections of this list and, for their section, produce a list of SVN > > > revisions (at the end of the checkme line) for which the PR number appears > > > to be correct, and a list of mappings from SVN revision to correct PR > > > number when the PR number appears to be wrong? For any that don't get > > > reviewed like that we can easily make the script, for the final > > > conversion, decline to add a new summary line for any commit where the PR > > > number is suspicious. > > > > Here's a slightly shorter version with 644 checkme: annotations, after > > adding a few more component aliases to the script (e.g., no longer > > considering it suspicious if the log message says PR g++/something and > > that PR is in the component that's actually called c++). > > Line 18: c++ SVN r116634, looks suspicious, but PR number is correct. > Line 326: lto SVN r196613, PR number is correct > Line 411: libstdc++ SVN r219147, PR number is correct > > > How do you want the mapping from SVN revision to correct PR to be expressed? > > Line 19: the correct PR for fortran SVN r120056 is fortran/30238 (not 39238) > Line 608: lto SVN r268728 should be PR 87089 (not 87809) > Line 616: lto SVN r269799 should be PR 87089 (not 87809) I can take a chunk of the file, but I'll wait until I know how to present the results.
Re: Commit messages and the move to git
On Thu, Dec 19, 2019 at 12:01:28AM +, Joseph Myers wrote: > re PR c/92324 (ICE in expand_direct_optab_fn, at internal-fn.c:2890 [checkme: > tree-optimization SVN r277822]) > re PR c/92324 (ICE in expand_direct_optab_fn, at internal-fn.c:2890 [checkme: > tree-optimization SVN r277955]) > re PR c/92324 (ICE in expand_direct_optab_fn, at internal-fn.c:2890 [checkme: > tree-optimization SVN r277958]) > re PR c/92324 (ICE in expand_direct_optab_fn, at internal-fn.c:2890 [checkme: > tree-optimization SVN r278289]) These are correct PR numbers, guess easiest check would be change c -> tree_optimization in the bugzilla to match what the bug really was. Jakub
Re: Commit messages and the move to git
On 19/12/2019 09:27, Jonathan Wakely wrote:
On Thu, 19 Dec 2019 at 00:02, Joseph Myers wrote:
On Wed, 18 Dec 2019, Joseph Myers wrote:
On Mon, 18 Nov 2019, Richard Earnshaw (lists) wrote:
I've attached a sample from the start of the fixed list - the full list is far
too big to post to give a flavour of how the script currently works. Note
that annotations of the form [checkme: ] in the summary are for diagnostic
purposes. These are where heuristics suggest that there's a higher than
normal chance that the PR number is incorrect and that manual auditing is
recommended. Such annotations would not be appropriate in the final
conversion.
Concretely, here is the current list of 664 checkme: annotations where
something was suspicious about the PR number (either component mismatch or
resolved as INVALID). Would some people like to volunteer to pick up
sections of this list and, for their section, produce a list of SVN
revisions (at the end of the checkme line) for which the PR number appears
to be correct, and a list of mappings from SVN revision to correct PR
number when the PR number appears to be wrong? For any that don't get
reviewed like that we can easily make the script, for the final
conversion, decline to add a new summary line for any commit where the PR
number is suspicious.
Here's a slightly shorter version with 644 checkme: annotations, after
adding a few more component aliases to the script (e.g., no longer
considering it suspicious if the log message says PR g++/something and
that PR is in the component that's actually called c++).
Line 18: c++ SVN r116634, looks suspicious, but PR number is correct.
Line 326: lto SVN r196613, PR number is correct
Line 411: libstdc++ SVN r219147, PR number is correct
How do you want the mapping from SVN revision to correct PR to be expressed?
Line 19: the correct PR for fortran SVN r120056 is fortran/30238 (not 39238)
Line 608: lto SVN r268728 should be PR 87089 (not 87809)
Line 616: lto SVN r269799 should be PR 87089 (not 87809)
Best of all would be a pull request on
https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py
directly.
Second best would be something like
whitelist:
"", "",
etc, where svn-revnumber is the revision number in svn as reported in
the checkme above but without the leading 'r'
and
Change:
"": {"PR": ""},
where svn-revnumber is as before, and is the the PR
number that should have been used.
The above can then be pasted quickly into the script to update it.
R.
Re: Commit messages and the move to git
On 19/12/2019 11:50, Richard Earnshaw (lists) wrote:
On 19/12/2019 09:27, Jonathan Wakely wrote:
On Thu, 19 Dec 2019 at 00:02, Joseph Myers
wrote:
On Wed, 18 Dec 2019, Joseph Myers wrote:
On Mon, 18 Nov 2019, Richard Earnshaw (lists) wrote:
I've attached a sample from the start of the fixed list - the full
list is far
too big to post to give a flavour of how the script currently
works. Note
that annotations of the form [checkme: ] in the summary are for
diagnostic
purposes. These are where heuristics suggest that there's a higher
than
normal chance that the PR number is incorrect and that manual
auditing is
recommended. Such annotations would not be appropriate in the final
conversion.
Concretely, here is the current list of 664 checkme: annotations where
something was suspicious about the PR number (either component
mismatch or
resolved as INVALID). Would some people like to volunteer to pick up
sections of this list and, for their section, produce a list of SVN
revisions (at the end of the checkme line) for which the PR number
appears
to be correct, and a list of mappings from SVN revision to correct PR
number when the PR number appears to be wrong? For any that don't get
reviewed like that we can easily make the script, for the final
conversion, decline to add a new summary line for any commit where
the PR
number is suspicious.
Here's a slightly shorter version with 644 checkme: annotations, after
adding a few more component aliases to the script (e.g., no longer
considering it suspicious if the log message says PR g++/something and
that PR is in the component that's actually called c++).
Line 18: c++ SVN r116634, looks suspicious, but PR number is correct.
Line 326: lto SVN r196613, PR number is correct
Line 411: libstdc++ SVN r219147, PR number is correct
How do you want the mapping from SVN revision to correct PR to be
expressed?
Line 19: the correct PR for fortran SVN r120056 is fortran/30238 (not
39238)
Line 608: lto SVN r268728 should be PR 87089 (not 87809)
Line 616: lto SVN r269799 should be PR 87089 (not 87809)
Best of all would be a pull request on
https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py
directly.
Second best would be something like
whitelist:
"", "",
etc, where svn-revnumber is the revision number in svn as reported in
the checkme above but without the leading 'r'
and
Change:
"": {"PR": ""},
There are two other actions that can be applied to commits, in the
change section:
"" {"ignore": True},
(case is important: True, not true or TRUE), will force the script to
ignore that commit entirely; you may end up with a pretty meaningless
summary line, so this should be used sparingly if at all.
and
"" {"summary": ""},
will use the specificed as the summary line.
Given the number of commits, we can't rewrite everything, but if you see
a summary that looks particularly wrong, we can insert something else
using this override.
R.
where svn-revnumber is as before, and is the the PR
number that should have been used.
The above can then be pasted quickly into the script to update it.
R.
Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 11:50, Richard Earnshaw (lists)
wrote:
>
> On 19/12/2019 09:27, Jonathan Wakely wrote:
> > On Thu, 19 Dec 2019 at 00:02, Joseph Myers wrote:
> >>
> >> On Wed, 18 Dec 2019, Joseph Myers wrote:
> >>
> >>> On Mon, 18 Nov 2019, Richard Earnshaw (lists) wrote:
> >>>
> I've attached a sample from the start of the fixed list - the full list
> is far
> too big to post to give a flavour of how the script currently works.
> Note
> that annotations of the form [checkme: ] in the summary are for
> diagnostic
> purposes. These are where heuristics suggest that there's a higher than
> normal chance that the PR number is incorrect and that manual auditing is
> recommended. Such annotations would not be appropriate in the final
> conversion.
> >>>
> >>> Concretely, here is the current list of 664 checkme: annotations where
> >>> something was suspicious about the PR number (either component mismatch or
> >>> resolved as INVALID). Would some people like to volunteer to pick up
> >>> sections of this list and, for their section, produce a list of SVN
> >>> revisions (at the end of the checkme line) for which the PR number appears
> >>> to be correct, and a list of mappings from SVN revision to correct PR
> >>> number when the PR number appears to be wrong? For any that don't get
> >>> reviewed like that we can easily make the script, for the final
> >>> conversion, decline to add a new summary line for any commit where the PR
> >>> number is suspicious.
> >>
> >> Here's a slightly shorter version with 644 checkme: annotations, after
> >> adding a few more component aliases to the script (e.g., no longer
> >> considering it suspicious if the log message says PR g++/something and
> >> that PR is in the component that's actually called c++).
> >
> > Line 18: c++ SVN r116634, looks suspicious, but PR number is correct.
> > Line 326: lto SVN r196613, PR number is correct
> > Line 411: libstdc++ SVN r219147, PR number is correct
> >
> >
> > How do you want the mapping from SVN revision to correct PR to be expressed?
> >
> > Line 19: the correct PR for fortran SVN r120056 is fortran/30238 (not 39238)
> > Line 608: lto SVN r268728 should be PR 87089 (not 87809)
> > Line 616: lto SVN r269799 should be PR 87089 (not 87809)
> >
>
> Best of all would be a pull request on
> https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py
> directly.
>
> Second best would be something like
>
> whitelist:
>
> "", "",
>
> etc, where svn-revnumber is the revision number in svn as reported in
> the checkme above but without the leading 'r'
>
> and
>
> Change:
>
> "": {"PR": ""},
>
>
> where svn-revnumber is as before, and is the the PR
> number that should have been used.
>
> The above can then be pasted quickly into the script to update it.
>
> R.
Thanks. I'm working through the first 100 lines in the file then.
If nobody else starts, I'll take the next 100, and so on.
Re: Commit messages and the move to git
On 19/12/2019 12:23, Jonathan Wakely wrote:
On Thu, 19 Dec 2019 at 11:50, Richard Earnshaw (lists)
wrote:
On 19/12/2019 09:27, Jonathan Wakely wrote:
On Thu, 19 Dec 2019 at 00:02, Joseph Myers wrote:
On Wed, 18 Dec 2019, Joseph Myers wrote:
On Mon, 18 Nov 2019, Richard Earnshaw (lists) wrote:
I've attached a sample from the start of the fixed list - the full list is far
too big to post to give a flavour of how the script currently works. Note
that annotations of the form [checkme: ] in the summary are for diagnostic
purposes. These are where heuristics suggest that there's a higher than
normal chance that the PR number is incorrect and that manual auditing is
recommended. Such annotations would not be appropriate in the final
conversion.
Concretely, here is the current list of 664 checkme: annotations where
something was suspicious about the PR number (either component mismatch or
resolved as INVALID). Would some people like to volunteer to pick up
sections of this list and, for their section, produce a list of SVN
revisions (at the end of the checkme line) for which the PR number appears
to be correct, and a list of mappings from SVN revision to correct PR
number when the PR number appears to be wrong? For any that don't get
reviewed like that we can easily make the script, for the final
conversion, decline to add a new summary line for any commit where the PR
number is suspicious.
Here's a slightly shorter version with 644 checkme: annotations, after
adding a few more component aliases to the script (e.g., no longer
considering it suspicious if the log message says PR g++/something and
that PR is in the component that's actually called c++).
Line 18: c++ SVN r116634, looks suspicious, but PR number is correct.
Line 326: lto SVN r196613, PR number is correct
Line 411: libstdc++ SVN r219147, PR number is correct
How do you want the mapping from SVN revision to correct PR to be expressed?
Line 19: the correct PR for fortran SVN r120056 is fortran/30238 (not 39238)
Line 608: lto SVN r268728 should be PR 87089 (not 87809)
Line 616: lto SVN r269799 should be PR 87089 (not 87809)
Best of all would be a pull request on
https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py
directly.
Second best would be something like
whitelist:
"", "",
etc, where svn-revnumber is the revision number in svn as reported in
the checkme above but without the leading 'r'
and
Change:
"": {"PR": ""},
where svn-revnumber is as before, and is the the PR
number that should have been used.
The above can then be pasted quickly into the script to update it.
R.
Thanks. I'm working through the first 100 lines in the file then.
If nobody else starts, I'll take the next 100, and so on.
Great, thanks.
It might be useful if someone could start from the other end. The later
numbers will be most recent and the ones which are more likely to be
relevant to users today.
My apologies that I'm not going to be able to contribute as much to this
as I'd hoped; events have conspired against me this week and today is
probably pretty much the only day that I'll be at a computer this side
of the new year.
R.
Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 12:33, Richard Earnshaw (lists)
wrote:
>
> On 19/12/2019 12:23, Jonathan Wakely wrote:
> > On Thu, 19 Dec 2019 at 11:50, Richard Earnshaw (lists)
> > wrote:
> >>
> >> On 19/12/2019 09:27, Jonathan Wakely wrote:
> >>> On Thu, 19 Dec 2019 at 00:02, Joseph Myers
> >>> wrote:
>
> On Wed, 18 Dec 2019, Joseph Myers wrote:
>
> > On Mon, 18 Nov 2019, Richard Earnshaw (lists) wrote:
> >
> >> I've attached a sample from the start of the fixed list - the full
> >> list is far
> >> too big to post to give a flavour of how the script currently works.
> >> Note
> >> that annotations of the form [checkme: ] in the summary are for
> >> diagnostic
> >> purposes. These are where heuristics suggest that there's a higher
> >> than
> >> normal chance that the PR number is incorrect and that manual auditing
> >> is
> >> recommended. Such annotations would not be appropriate in the final
> >> conversion.
> >
> > Concretely, here is the current list of 664 checkme: annotations where
> > something was suspicious about the PR number (either component mismatch
> > or
> > resolved as INVALID). Would some people like to volunteer to pick up
> > sections of this list and, for their section, produce a list of SVN
> > revisions (at the end of the checkme line) for which the PR number
> > appears
> > to be correct, and a list of mappings from SVN revision to correct PR
> > number when the PR number appears to be wrong? For any that don't get
> > reviewed like that we can easily make the script, for the final
> > conversion, decline to add a new summary line for any commit where the
> > PR
> > number is suspicious.
>
> Here's a slightly shorter version with 644 checkme: annotations, after
> adding a few more component aliases to the script (e.g., no longer
> considering it suspicious if the log message says PR g++/something and
> that PR is in the component that's actually called c++).
> >>>
> >>> Line 18: c++ SVN r116634, looks suspicious, but PR number is correct.
> >>> Line 326: lto SVN r196613, PR number is correct
> >>> Line 411: libstdc++ SVN r219147, PR number is correct
> >>>
> >>>
> >>> How do you want the mapping from SVN revision to correct PR to be
> >>> expressed?
> >>>
> >>> Line 19: the correct PR for fortran SVN r120056 is fortran/30238 (not
> >>> 39238)
> >>> Line 608: lto SVN r268728 should be PR 87089 (not 87809)
> >>> Line 616: lto SVN r269799 should be PR 87089 (not 87809)
> >>>
> >>
> >> Best of all would be a pull request on
> >> https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py
> >> directly.
> >>
> >> Second best would be something like
> >>
> >> whitelist:
> >>
> >> "", "",
> >>
> >> etc, where svn-revnumber is the revision number in svn as reported in
> >> the checkme above but without the leading 'r'
> >>
> >> and
> >>
> >> Change:
> >>
> >> "": {"PR": ""},
> >>
> >>
> >> where svn-revnumber is as before, and is the the PR
> >> number that should have been used.
> >>
> >> The above can then be pasted quickly into the script to update it.
> >>
> >> R.
> >
> > Thanks. I'm working through the first 100 lines in the file then.
> >
> > If nobody else starts, I'll take the next 100, and so on.
> >
>
> Great, thanks.
>
> It might be useful if someone could start from the other end. The later
> numbers will be most recent and the ones which are more likely to be
> relevant to users today.
And less likely to refer to egcs bugs and/or egcs patches from 1997
which are harder to find in our mailing lists archives!
Since nobody else has volunteered yet, maybe I should just start at
the end instead. All I've managed so far is to decide that line 1 is
too hard to track down and should get a {"summary":"..."} fixup
instead.
So I'll start with the LAST 100 lines.
Re: Commit messages and the move to git
On 19/12/2019 12:35, Jonathan Wakely wrote:
On Thu, 19 Dec 2019 at 12:33, Richard Earnshaw (lists)
wrote:
On 19/12/2019 12:23, Jonathan Wakely wrote:
On Thu, 19 Dec 2019 at 11:50, Richard Earnshaw (lists)
wrote:
On 19/12/2019 09:27, Jonathan Wakely wrote:
On Thu, 19 Dec 2019 at 00:02, Joseph Myers wrote:
On Wed, 18 Dec 2019, Joseph Myers wrote:
On Mon, 18 Nov 2019, Richard Earnshaw (lists) wrote:
I've attached a sample from the start of the fixed list - the full list is far
too big to post to give a flavour of how the script currently works. Note
that annotations of the form [checkme: ] in the summary are for diagnostic
purposes. These are where heuristics suggest that there's a higher than
normal chance that the PR number is incorrect and that manual auditing is
recommended. Such annotations would not be appropriate in the final
conversion.
Concretely, here is the current list of 664 checkme: annotations where
something was suspicious about the PR number (either component mismatch or
resolved as INVALID). Would some people like to volunteer to pick up
sections of this list and, for their section, produce a list of SVN
revisions (at the end of the checkme line) for which the PR number appears
to be correct, and a list of mappings from SVN revision to correct PR
number when the PR number appears to be wrong? For any that don't get
reviewed like that we can easily make the script, for the final
conversion, decline to add a new summary line for any commit where the PR
number is suspicious.
Here's a slightly shorter version with 644 checkme: annotations, after
adding a few more component aliases to the script (e.g., no longer
considering it suspicious if the log message says PR g++/something and
that PR is in the component that's actually called c++).
Line 18: c++ SVN r116634, looks suspicious, but PR number is correct.
Line 326: lto SVN r196613, PR number is correct
Line 411: libstdc++ SVN r219147, PR number is correct
How do you want the mapping from SVN revision to correct PR to be expressed?
Line 19: the correct PR for fortran SVN r120056 is fortran/30238 (not 39238)
Line 608: lto SVN r268728 should be PR 87089 (not 87809)
Line 616: lto SVN r269799 should be PR 87089 (not 87809)
Best of all would be a pull request on
https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py
directly.
Second best would be something like
whitelist:
"", "",
etc, where svn-revnumber is the revision number in svn as reported in
the checkme above but without the leading 'r'
and
Change:
"": {"PR": ""},
where svn-revnumber is as before, and is the the PR
number that should have been used.
The above can then be pasted quickly into the script to update it.
R.
Thanks. I'm working through the first 100 lines in the file then.
If nobody else starts, I'll take the next 100, and so on.
Great, thanks.
It might be useful if someone could start from the other end. The later
numbers will be most recent and the ones which are more likely to be
relevant to users today.
And less likely to refer to egcs bugs and/or egcs patches from 1997
which are harder to find in our mailing lists archives!
Since nobody else has volunteered yet, maybe I should just start at
the end instead. All I've managed so far is to decide that line 1 is
too hard to track down and should get a {"summary":"..."} fixup
instead.
So I'll start with the LAST 100 lines.
Another approach might be to do a quick triage and cull out (whitelist)
the ones that are "obviously correct". You can often tell by reading
the summary itself that it really does correspond to the commit. Then
we'd be left with a shorter list where things really do need further
digging. In the worst case we can just do as Joseph suggests and
implement a policy ignore for those in the final conversion (I already
do that for PR numbers <1000 since there was more than one gnats DB at
that time and the PR numbers just do not line up reliably enough).
R.
Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 12:42, Richard Earnshaw (lists)
wrote:
>
> On 19/12/2019 12:35, Jonathan Wakely wrote:
> > On Thu, 19 Dec 2019 at 12:33, Richard Earnshaw (lists)
> > wrote:
> >>
> >> On 19/12/2019 12:23, Jonathan Wakely wrote:
> >>> On Thu, 19 Dec 2019 at 11:50, Richard Earnshaw (lists)
> >>> wrote:
>
> On 19/12/2019 09:27, Jonathan Wakely wrote:
> > On Thu, 19 Dec 2019 at 00:02, Joseph Myers
> > wrote:
> >>
> >> On Wed, 18 Dec 2019, Joseph Myers wrote:
> >>
> >>> On Mon, 18 Nov 2019, Richard Earnshaw (lists) wrote:
> >>>
> I've attached a sample from the start of the fixed list - the full
> list is far
> too big to post to give a flavour of how the script currently works.
> Note
> that annotations of the form [checkme: ] in the summary are for
> diagnostic
> purposes. These are where heuristics suggest that there's a higher
> than
> normal chance that the PR number is incorrect and that manual
> auditing is
> recommended. Such annotations would not be appropriate in the final
> conversion.
> >>>
> >>> Concretely, here is the current list of 664 checkme: annotations where
> >>> something was suspicious about the PR number (either component
> >>> mismatch or
> >>> resolved as INVALID). Would some people like to volunteer to pick up
> >>> sections of this list and, for their section, produce a list of SVN
> >>> revisions (at the end of the checkme line) for which the PR number
> >>> appears
> >>> to be correct, and a list of mappings from SVN revision to correct PR
> >>> number when the PR number appears to be wrong? For any that don't get
> >>> reviewed like that we can easily make the script, for the final
> >>> conversion, decline to add a new summary line for any commit where
> >>> the PR
> >>> number is suspicious.
> >>
> >> Here's a slightly shorter version with 644 checkme: annotations, after
> >> adding a few more component aliases to the script (e.g., no longer
> >> considering it suspicious if the log message says PR g++/something and
> >> that PR is in the component that's actually called c++).
> >
> > Line 18: c++ SVN r116634, looks suspicious, but PR number is correct.
> > Line 326: lto SVN r196613, PR number is correct
> > Line 411: libstdc++ SVN r219147, PR number is correct
> >
> >
> > How do you want the mapping from SVN revision to correct PR to be
> > expressed?
> >
> > Line 19: the correct PR for fortran SVN r120056 is fortran/30238 (not
> > 39238)
> > Line 608: lto SVN r268728 should be PR 87089 (not 87809)
> > Line 616: lto SVN r269799 should be PR 87089 (not 87809)
> >
>
> Best of all would be a pull request on
> https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py
> directly.
>
> Second best would be something like
>
> whitelist:
>
> "", "",
>
> etc, where svn-revnumber is the revision number in svn as reported in
> the checkme above but without the leading 'r'
>
> and
>
> Change:
>
> "": {"PR": ""},
>
>
> where svn-revnumber is as before, and is the the PR
> number that should have been used.
>
> The above can then be pasted quickly into the script to update it.
>
> R.
> >>>
> >>> Thanks. I'm working through the first 100 lines in the file then.
> >>>
> >>> If nobody else starts, I'll take the next 100, and so on.
> >>>
> >>
> >> Great, thanks.
> >>
> >> It might be useful if someone could start from the other end. The later
> >> numbers will be most recent and the ones which are more likely to be
> >> relevant to users today.
> >
> > And less likely to refer to egcs bugs and/or egcs patches from 1997
> > which are harder to find in our mailing lists archives!
> >
> > Since nobody else has volunteered yet, maybe I should just start at
> > the end instead. All I've managed so far is to decide that line 1 is
> > too hard to track down and should get a {"summary":"..."} fixup
> > instead.
> >
> > So I'll start with the LAST 100 lines.
> >
>
> Another approach might be to do a quick triage and cull out (whitelist)
> the ones that are "obviously correct". You can often tell by reading
> the summary itself that it really does correspond to the commit. Then
> we'd be left with a shorter list where things really do need further
> digging. In the worst case we can just do as Joseph suggests and
> implement a policy ignore for those in the final conversion (I already
> do that for PR numbers <1000 since there was more than one gnats DB at
> that time and the PR numbers just do not line up reliably enough).
>
> R.
It might be reasonable to assume rtl-optimization and
tree-optimization are aliases, and not treat it as sus
Re: Test GCC conversions (publicly) available
Hi, On Wed, 2019-12-18 at 21:55 +, Joseph Myers wrote: > On Tue, 17 Dec 2019, Joseph Myers wrote: > > I've made test conversions of the GCC repository with reposurgeon > > available (gcc.gnu.org / sourceware.org account required to access > > these git+ssh repositories, it doesn't need to be one in the gcc group > > or to have shell access). More information about the repositories, > > conversion choices made and known issues is given below, and, as noted > > there, I'm running another conversion now with fixes for some of those > > issues and the remaining listed issues not fixed in that conversion > > are being actively worked on. > > > > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-1a.git > > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-1b.git > > There are now four more repositories available. > > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-2a.git > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-2b.git > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-3a.git > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-3b.git For those without gcc.gnu.org accounts I made mirrors at: https://code.wildebeest.org/git/gcc/ There is also a mirror of Maxim's gcc-reparent branch: https://code.wildebeest.org/git/gcc/gcc-reparent/ And the traditional gcc git-svn mirror: https://code.wildebeest.org/git/mirror/gcc/ The last two seem to be kept up to date with current svn. Cloning should work through both git:// and (smart) https:// protocols. Please be gentle. This is on a little machine in my basement. Each full clone also takes 1GB+ of data. So you might just want to look at them just through the cgit webinterface. But hopefully it is still useful. I will likely delete these again after we have picked a conversion we actually want to use. Do we already have a new date for when we are making that decision? Thanks, Mark
Re: Test GCC conversions (publicly) available
Mark Wielaard : > Do we already have a new date for when we are making that decision? I believe Joseph was planning on Dec 31st. My team's part will be ready - the enabling reposurgeon changes should done in a week or so, with most of that being RFEs that could be dropped if there were real time pressure. There are other problems that might cause a delay beyond the 31st, however. Best if I let Joseph nd Richard explain those. -- http://www.catb.org/~esr/";>Eric S. Raymond
Re: Commit messages and the move to git
On Thu, 19 Dec 2019, Richard Earnshaw (lists) wrote: > Best of all would be a pull request on > https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py directly. Note if doing that, it helps to check "Allow commits from members who can merge to the target branch." when creating the merge request (that allows using GitLab's automated rebasing when merging a merge request created against a revision of master that's no longer the most recent revision). -- Joseph S. Myers jos...@codesourcery.com
Re: Test GCC conversions (publicly) available
On Thu, 19 Dec 2019, Eric S. Raymond wrote: > There are other problems that might cause a delay beyond the > 31st, however. Best if I let Joseph nd Richard explain those. I presume that's referring to the checkme: bug annotations where the PR numbers in commit messages seem suspicious. I don't think that's something to delay the conversion unless we're clearly close to having a complete review of all those cases done; at the point of the final conversion we should simply change the script not to modify commit messages in any remaining unresolved suspicious cases. -- Joseph S. Myers jos...@codesourcery.com
Re: Commit messages and the move to git
On 19/12/2019 11:16, Jakub Jelinek wrote: On Thu, Dec 19, 2019 at 12:01:28AM +, Joseph Myers wrote: re PR c/92324 (ICE in expand_direct_optab_fn, at internal-fn.c:2890 [checkme: tree-optimization SVN r277822]) re PR c/92324 (ICE in expand_direct_optab_fn, at internal-fn.c:2890 [checkme: tree-optimization SVN r277955]) re PR c/92324 (ICE in expand_direct_optab_fn, at internal-fn.c:2890 [checkme: tree-optimization SVN r277958]) re PR c/92324 (ICE in expand_direct_optab_fn, at internal-fn.c:2890 [checkme: tree-optimization SVN r278289]) These are correct PR numbers, guess easiest check would be change c -> tree_optimization in the bugzilla to match what the bug really was. Jakub Yes, we can certainly fix the bugzilla entries if that's easier. To pick the changes up we'd have to rebuild the cache that my scripts use to avoid hammering the bugzilla service, but Joseph can do that now successfully, so it shouldn't be a problem as long as we know that needs doing. R. [not using the cache risks the PR data not being pulled properly during a conversion run due to network or load-dependent issues on the gcc server].
Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 14:29, Joseph Myers wrote: > > On Thu, 19 Dec 2019, Richard Earnshaw (lists) wrote: > > > Best of all would be a pull request on > > https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py > > directly. > > Note if doing that, it helps to check "Allow commits from members who can > merge to the target branch." when creating the merge request (that allows > using GitLab's automated rebasing when merging a merge request created > against a revision of master that's no longer the most recent revision). Done. https://gitlab.com/esr/gcc-conversion/merge_requests/25 fixes (most of?) the most egregious ones, like fortran fixes with c++ PR numbers and vice versa. Jakub and I have several whitelist commits too, but I think they're all less critical (as mentioned previously, most of them are unimportant differences like rtl-optimization vs tree-optimization, or debug vs lto, or simply that the Bugzilla component has been changed since the commit was done and the commit was actually right at the time).
Re: Commit messages and the move to git
On 19/12/2019 15:17, Jonathan Wakely wrote: On Thu, 19 Dec 2019 at 14:29, Joseph Myers wrote: On Thu, 19 Dec 2019, Richard Earnshaw (lists) wrote: Best of all would be a pull request on https://gitlab.com/esr/gcc-conversion/tree/master to update bugdb.py directly. Note if doing that, it helps to check "Allow commits from members who can merge to the target branch." when creating the merge request (that allows using GitLab's automated rebasing when merging a merge request created against a revision of master that's no longer the most recent revision). Done. https://gitlab.com/esr/gcc-conversion/merge_requests/25 fixes (most of?) the most egregious ones, like fortran fixes with c++ PR numbers and vice versa. Jakub and I have several whitelist commits too, but I think they're all less critical (as mentioned previously, most of them are unimportant differences like rtl-optimization vs tree-optimization, or debug vs lto, or simply that the Bugzilla component has been changed since the commit was done and the commit was actually right at the time). Merged
Re: Commit messages and the move to git
These scraped "INVALID" as the component from the changelog, because it said "libgfortran/24685": revert: re PR libfortran/24685 (real(16) formatted input is broken for huge values (gfortran.dg/default_format_2.f90) [checkme: INVALID SVN r142840]) revert: re PR libfortran/24685 (real(16) formatted input is broken for huge values (gfortran.dg/default_format_2.f90) [checkme: INVALID SVN r142841]) Shouldn't that be handled by the "compalias" dict? Why are these invalid? The changelogs look OK: re PR c++/46134 (constexpr vs. defaulted ctor [checkme: INVALID SVN r166171]) re PR c/11586 (after call sigaction, system() return wrong status [checkme: INVALID SVN r187542]) re PR target/86677 (popcount builtin detection is breaking some kernel build [checkme: INVALID SVN r266039])
Re: Commit messages and the move to git
On Thu, 19 Dec 2019, Jonathan Wakely wrote: > These scraped "INVALID" as the component from the changelog, because > it said "libgfortran/24685": "INVALID" means the PR was closed as INVALID rather than FIXED, which makes it suspect for a commit to claim to be fixing it. (Though since those are marked as "revert", they're probably OK and should be whitelisted.) -- Joseph S. Myers jos...@codesourcery.com
Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 15:47, Joseph Myers wrote: > > On Thu, 19 Dec 2019, Jonathan Wakely wrote: > > > These scraped "INVALID" as the component from the changelog, because > > it said "libgfortran/24685": > > "INVALID" means the PR was closed as INVALID rather than FIXED, which > makes it suspect for a commit to claim to be fixing it. (Though since > those are marked as "revert", they're probably OK and should be > whitelisted.) Aha, thanks.
Re: Commit messages and the move to git
On 19/12/2019 15:44, Jonathan Wakely wrote: These scraped "INVALID" as the component from the changelog, because it said "libgfortran/24685": revert: re PR libfortran/24685 (real(16) formatted input is broken for huge values (gfortran.dg/default_format_2.f90) [checkme: INVALID SVN r142840]) revert: re PR libfortran/24685 (real(16) formatted input is broken for huge values (gfortran.dg/default_format_2.f90) [checkme: INVALID SVN r142841]) Shouldn't that be handled by the "compalias" dict? Why are these invalid? The changelogs look OK: re PR c++/46134 (constexpr vs. defaulted ctor [checkme: INVALID SVN r166171]) re PR c/11586 (after call sigaction, system() return wrong status [checkme: INVALID SVN r187542]) re PR target/86677 (popcount builtin detection is breaking some kernel build [checkme: INVALID SVN r266039]) INVALID means the PR was closed as invalid, which seems an suspect closure state for a PR that has had a patch installed. In some cases the bug really was invalid and the commit is just some doc or other type cleanup, but it's often hard to be sure in the automated code, so it's flagged up for manual checking. R.
Re: Commit messages and the move to git
On Thu, 19 Dec 2019, Richard Earnshaw (lists) wrote: > > Done. https://gitlab.com/esr/gcc-conversion/merge_requests/25 fixes > > (most of?) the most egregious ones, like fortran fixes with c++ PR > > numbers and vice versa. Jakub and I have several whitelist commits > > too, but I think they're all less critical (as mentioned previously, > > most of them are unimportant differences like rtl-optimization vs > > tree-optimization, or debug vs lto, or simply that the Bugzilla > > component has been changed since the commit was done and the commit > > was actually right at the time). > > > > Merged And I've fixed up some Python syntax there (commas that needed to be colons, between the commit number and the fixup information). -- Joseph S. Myers jos...@codesourcery.com
Re: Test GCC conversion with reposurgeon available
On Thu, 19 Dec 2019, Jason Merrill wrote: > So a 30% space savings; that's pretty significant. Though I wonder how > much of that is refs/dead and refs/deleted, which seem unnecessary to carry > over to git at all. I wonder if it would make sense to put them in a > separate repository that refers to the main gcc.git? refs/dead is definitely relevant sometimes; that's old development branches. refs/deleted is less clearly relevant. -- Joseph S. Myers jos...@codesourcery.com
Re: Test GCC conversions (publicly) available
Joseph Myers : > On Thu, 19 Dec 2019, Eric S. Raymond wrote: > > > There are other problems that might cause a delay beyond the > > 31st, however. Best if I let Joseph nd Richard explain those. > > I presume that's referring to the checkme: bug annotations where the PR > numbers in commit messages seem suspicious. I don't think that's > something to delay the conversion unless we're clearly close to having a > complete review of all those cases done; at the point of the final > conversion we should simply change the script not to modify commit > messages in any remaining unresolved suspicious cases. No, I was thinking more of rearnsha bailing out to handle a family emergency and muttering something about not being back for a couple of weeks. If that's been resolved I haven't heard about it. The only conversion blocker that I know is still live is the wrong attributions in some ChangeLog cases. I'm sure we'll get that fixed soon; at this point I'm more worried about getting the test suite to run clean again. The scenario I want to avoid is the where you get a conversion that looks production-ready before I get my tests cleaned up, you deploy it - and then I find something during the remainder of my cleanup that implies a problem with your conversion. A complicating factor is that I'm getting stale. I've been going hammer and tongs at this for nearly three months now, and that's not counting all the previous time on the Go translation. My defect rate is going up. I need a vacation or to work on something else for a while and I can't have that yet. Never nind. We'll get this done. -- http://www.catb.org/~esr/";>Eric S. Raymond
Re: Commit messages and the move to git
On Thu, 19 Dec 2019, Jonathan Wakely wrote: > It might be reasonable to assume rtl-optimization and > tree-optimization are aliases, and not treat it as suspicious if those > two appear mixed up. And anything where bugzilla has component debug > or lto and the commit is tree-optimization is probably OK too (that > seems to be the case for several so far). I've added those aliases (and the "tree-optimizatin" typo that appears a few times). The case of (lto, tree-optimization) was already listed as an alias. -- Joseph S. Myers jos...@codesourcery.com
Re: Test GCC conversions (publicly) available
On 19/12/2019 16:00, Eric S. Raymond wrote: Joseph Myers : On Thu, 19 Dec 2019, Eric S. Raymond wrote: There are other problems that might cause a delay beyond the 31st, however. Best if I let Joseph nd Richard explain those. I presume that's referring to the checkme: bug annotations where the PR numbers in commit messages seem suspicious. I don't think that's something to delay the conversion unless we're clearly close to having a complete review of all those cases done; at the point of the final conversion we should simply change the script not to modify commit messages in any remaining unresolved suspicious cases. No, I was thinking more of rearnsha bailing out to handle a family emergency and muttering something about not being back for a couple of weeks. If that's been resolved I haven't heard about it. I don't think that should affect things, as I think Joseph has a good handle on what needs to be done and I think I've handed over everything that's needed w.r.t. the commit summary reprocessing script. Joseph, holler quickly if you think there's anything else you might need... R.
Re: Commit messages and the move to git
On 19/12/2019 16:00, Joseph Myers wrote: On Thu, 19 Dec 2019, Jonathan Wakely wrote: It might be reasonable to assume rtl-optimization and tree-optimization are aliases, and not treat it as suspicious if those two appear mixed up. And anything where bugzilla has component debug or lto and the commit is tree-optimization is probably OK too (that seems to be the case for several so far). I've added those aliases (and the "tree-optimizatin" typo that appears a few times). The case of (lto, tree-optimization) was already listed as an alias. Aliases are uni-directional. You might need the reverse as well. R.
Re: Test GCC conversions (publicly) available
Richard Earnshaw (lists) : > > No, I was thinking more of rearnsha bailing out to handle a family emergency > > and muttering something about not being back for a couple of weeks. If > > that's > > been resolved I haven't heard about it. > > I don't think that should affect things, as I think Joseph has a good handle > on what needs to be done and I think I've handed over everything that's > needed w.r.t. the commit summary reprocessing script. OK, that's good to know. I wish you good fortune with the emergency. -- http://www.catb.org/~esr/";>Eric S. Raymond
Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 15:49, Joseph Myers wrote: > > On Thu, 19 Dec 2019, Richard Earnshaw (lists) wrote: > > > > Done. https://gitlab.com/esr/gcc-conversion/merge_requests/25 fixes > > > (most of?) the most egregious ones, like fortran fixes with c++ PR > > > numbers and vice versa. Jakub and I have several whitelist commits > > > too, but I think they're all less critical (as mentioned previously, > > > most of them are unimportant differences like rtl-optimization vs > > > tree-optimization, or debug vs lto, or simply that the Bugzilla > > > component has been changed since the commit was done and the commit > > > was actually right at the time). > > > > > > > Merged > > And I've fixed up some Python syntax there (commas that needed to be > colons, between the commit number and the fixup information). Thanks. I'm afraid it was too mind-numbing to just work through the list from bottom to top. Jakub and I found some problems and some that can be whitelisted. I submitted another merge request. I've attached an updated list to this mail, which removes the items we've analysed. There are 531 remaining. re PR rtl-optimization/13024 (gcj can't build current rhug [checkme: java SVN r73752]) backport: re PR rtl-optimization/12816 (internal compiler error pari-2.1.5/Olinux-i686 [checkme: c++ SVN r75851]) revert: re PR tree-optimization/16115 (double-destruction problem with argument passing via temporary (breaks auto_ptr) [checkme: c++ SVN r84147]) re PR tree-optimization/15262 ([tree-ssa] Alias analyzer cannot handle addressable fields [checkme: c SVN r86398]) re PR rtl-optimization/15857 (Wrong code with optimization >= -O1 [checkme: c++ SVN r87429]) re PR c/16403 (Floating-point assignment to int is inconsistent [checkme: INVALID SVN r94142]) re PR c++/20505 (internal error when compiling with -ggdb2 and no error with -ggdb1 [checkme: debug SVN r97528]) re PR rtl-optimization/19210 (not using do-loop for some loops [checkme: tree-optimization SVN r102225]) re PR tree-optimization/21562 (Quiet bad codegen (unrolling + tail call interaction) [checkme: c SVN r103245]) re PR c/21419 (Accepts writting to const via asm [checkme: tree-optimization SVN r104991]) re PR awt/26641 (AWT mouse event handling regression [checkme: libgcj SVN r112464]) re PR java/28024 (libjava build failure on Solaris 2.8 (sun4u) [checkme: INVALID SVN r114637]) re PR java/28024 (libjava build failure on Solaris 2.8 (sun4u) [checkme: INVALID SVN r114639]) re PR middle-end/25505 (gcc uses way too much stack space for this code [checkme: c++ SVN r116634]) re PR libstdc++/39238 (trunk revision 144279 - cfenv:54: error: ‘::fenv_t’ has not been declared [checkme: fortran SVN r120056]) re PR driver/30714 (gcj driver doesn't recognize files starting with II [checkme: java SVN r121666]) re PR driver/30714 (gcj driver doesn't recognize files starting with II [checkme: java SVN r121667]) re PR debug/33739 (Failure of gfortran.dg/literal_character_constant_1_*.F with -m64 -g on Darwin [checkme: fortran SVN r130244]) re PR c++/31863 (g++-4.1: out of memory with -O1/-O2 [checkme: middle-end SVN r131405]) re PR c/34601 (ICE with undefined enum [checkme: middle-end SVN r131506]) re PR middle-end/34668 (ICE in find_compatible_field with -combine [checkme: c SVN r131572]) re PR tree-optimization/26854 (Inordinate compile times on large routines [checkme: rtl-optimization SVN r131649]) re PR tree-optimization/26854 (Inordinate compile times on large routines [checkme: rtl-optimization SVN r131670]) re PR tree-optimization/34885 (ICE in copy_reference_ops_from_ref, at tree-ssa-sccvn.c:574 [checkme: c SVN r131694]) re PR tree-optimization/26854 (Inordinate compile times on large routines [checkme: rtl-optimization SVN r131719]) re PR c++/34953 (ICE on destructor + noreturn-function at -O3 [checkme: middle-end SVN r131782]) re PR translation/35002 (Incorrect spelling of "hottest" [checkme: c SVN r131940]) re PR driver/30330 (-Wdeprecated is not documented [checkme: documentation SVN r132131]) re PR tree-optimization/33512 (Simple bitwise simplification missed [checkme: rtl-opt SVN r132575]) re PR c/35526 (ICE on memcpy [checkme: middle-end SVN r133106]) re PR c/35526 (ICE on memcpy [checkme: middle-end SVN r133108]) re PR preprocessor/35322 (ICE with incomplete macro [checkme: libcpp SVN r133195]) re PR preprocessor/35322 (ICE with incomplete macro [checkme: libcpp SVN r133220]) re PR rtl-optimization/19580 (missed load/store motion [checkme: tree-optimization SVN r133683]) re PR preprocessor/34866 (valgrind error indication in testsuite from errors.c:156:cpp_error with gcc.dg/cpp/Wmissingdirs.c [checkme: libcpp SVN r134421]) re PR preprocessor/15500 (gcc ignores locale when converting wide string literals to wchar_t strings [checkme: libcpp SVN r134441]) re PR preprocessor/33415 (Can't compile .cpp file with UTF-8 BOM. [checkme: libcpp SVN r134507]) re PR c++/35652 (offset warning should be given in the front-end [checkm
Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 16:26, Jonathan Wakely wrote: > > On Thu, 19 Dec 2019 at 15:49, Joseph Myers wrote: > > > > On Thu, 19 Dec 2019, Richard Earnshaw (lists) wrote: > > > > > > Done. https://gitlab.com/esr/gcc-conversion/merge_requests/25 fixes > > > > (most of?) the most egregious ones, like fortran fixes with c++ PR > > > > numbers and vice versa. Jakub and I have several whitelist commits > > > > too, but I think they're all less critical (as mentioned previously, > > > > most of them are unimportant differences like rtl-optimization vs > > > > tree-optimization, or debug vs lto, or simply that the Bugzilla > > > > component has been changed since the commit was done and the commit > > > > was actually right at the time). > > > > > > > > > > Merged > > > > And I've fixed up some Python syntax there (commas that needed to be > > colons, between the commit number and the fixup information). > > Thanks. > > I'm afraid it was too mind-numbing to just work through the list from > bottom to top. Jakub and I found some problems To be clear, by "problems" I mean entries that were indeed wrong and need a fixup.
Re: Test GCC conversion with reposurgeon available
On Wed, 18 Dec 2019, Joseph Myers wrote: > There are now four more repositories available. > > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-2a.git > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-2b.git > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-3a.git > git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-3b.git And two more. git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-4a.git git+ssh://gcc.gnu.org/home/gccadmin/gcc-reposurgeon-4b.git The main changes in this version are: * More mergeinfo improvements, so more valid merges should be detected and represented as such in git. (This doesn't yet have the changes to handle the case of both svnmerge-integrated and svn:mergeinfo having relevant merge information.) * More bug data used with Richard's script (but not the most recent whitelisting / PR corrections). * @gcc.gnu.org / @gnu.org addresses are preferred in the author map, to avoid anachronistic credits of commits to addresses at the wrong company (this also means that the git "committer" information consistently uses such addresses, which is certainly logically correct). I preserved timezone annotations in the map for existing addresses but accidentally did that in a way that resulted in those existing addresses, when used in ChangeLog entries, being mapped to the @gcc.gnu.org / @gnu.org ones; I'll fix that for the next conversion run so the addresses from the ChangeLog entries are properly preserved in such cases but still use the timezone annotations from gcc.map. -- Joseph S. Myers jos...@codesourcery.com
Re: Commit messages and the move to git
On Thu, 19 Dec 2019, Jonathan Wakely wrote: > On Thu, 19 Dec 2019 at 15:49, Joseph Myers wrote: > > > > On Thu, 19 Dec 2019, Richard Earnshaw (lists) wrote: > > > > > > Done. https://gitlab.com/esr/gcc-conversion/merge_requests/25 fixes > > > > (most of?) the most egregious ones, like fortran fixes with c++ PR > > > > numbers and vice versa. Jakub and I have several whitelist commits > > > > too, but I think they're all less critical (as mentioned previously, > > > > most of them are unimportant differences like rtl-optimization vs > > > > tree-optimization, or debug vs lto, or simply that the Bugzilla > > > > component has been changed since the commit was done and the commit > > > > was actually right at the time). > > > > > > > > > > Merged > > > > And I've fixed up some Python syntax there (commas that needed to be > > colons, between the commit number and the fixup information). > > Thanks. > > I'm afraid it was too mind-numbing to just work through the list from > bottom to top. Jakub and I found some problems and some that can be > whitelisted. I submitted another merge request. Thanks, merged. If people have done updates to components in Bugzilla and want a fresh list generated using fresh Bugzilla data and the updates to component aliases, I can readily do that (using the file of commit messages left behind from my last conversion run), load on sourceware permitting. -- Joseph S. Myers jos...@codesourcery.com
Re: Struct declaration and initialization in TREES
On 2019-12-18 1:33 p.m., Erick Ochoa wrote:
>
>
> On 2019-12-18 6:02 a.m., Richard Biener wrote:
>> On December 17, 2019 8:31:00 PM GMT+01:00, Erick Ochoa
>> wrote:
>>> Hi,
>>>
>>> I'm interested in printing VAR_DECL trees that are of type
>>> RECORD_TYPE. I am using the function print_generic_decl
>>> for debugging and found this interesting behaviour.
>>>
>>> When I do initialization of structs using the following
>>> syntax:
>>>
>>> ```
>>> struct aStruct { _Bool e; int a; char b; float c; double d; _Bool f;
>>> long h;};
>>> struct aStruct astruct = { .e = 0, .a = 1, .b = 2, .c = 3.0, .d = 4.0,
>>> .f = 5, .h = 6 };
>>> ```
>>>
>>> print_generic_decl will print the following output:
>>>
>>> ```
>>> INTERESTED IN: static struct aStruct astruct = <<< error >>>;
>>> ```
>>>
>>> Am I using the correct API? Is this a limitation of the API?
>>
>> You need to ask IPA about the static initializer, DECL_INITIAL is adjusted
>> at some point.
>>
>> Richard.
>
> Thanks Richard.
>
> I still don't understand very well what I should do. I found the
> place in GCC internals where DECL_INITIAL is described.
>
> https://gcc.gnu.org/onlinedocs/gccint/Working-with-declarations.html
>
> VAR_DECL
>
> If this variable is initialized (but does not require a constructor),
> the DECL_INITIAL will be an expression for the initializer.
> The initializer should be evaluated, and a bitwise copy into the variable
> performed. If the DECL_INITIAL is the error_mark_node, there is an
> initializer,
> but it is given by an explicit statement later in the code; no bitwise copy
> is required.
>
> So, now I understand that I can now change my patch to something like this:
>
>>> +iphw_execute()
>>> +{
>>> + if (!dump_file) return 0;
>>> + varpool_node *node;
>>> + FOR_EACH_VARIABLE(node)
>>> + {
>>> +gcc_assert(node->decl);
>>> +tree decl = node->decl;
>>> +gcc_assert(TREE_CODE(decl) == VAR_DECL);
>>> +
>>> +tree curr_type = TREE_TYPE(decl);
>>> +gcc_assert(curr_type);
>>> +
>>> +const bool is_struct = TREE_CODE(curr_type) == RECORD_TYPE;
>>> +if (is_struct) fprintf(dump_file, "INTERESTED IN: ");
>
> tree expr = DECL_INITIAL(decl);
> if (expr == error_mark_node) {
>fprintf(dump_file, "initializer is given by explicit statement
> later in code\n");
>continue;
> }
>
>>> +print_generic_decl(dump_file, decl, TDF_DETAILS);
>>> +fprintf(dump_file, "\n");
>>> + }
>
> However I still don't know how to ask IPA about the static initializer.
> I tried iterating over the ipa_refs (i.e. node->iterate_referring (i, ref))
> and then looking for writes, but there are now writes because it is a
> initialized statically.
I found some more documentation about static initializers
during LTO. In GCC internals it says:
https://gcc.gnu.org/onlinedocs/gccint/LTO-object-file-layout.html#LTO-object-file-layout
Static variable initializers (.gnu.lto_.vars)
This section contains all the symbols in the global variable pool. It is
emitted by lto-cgraph.c:output_varpool and read in lto-cgraph.c:input_cgraph.
Does this mean that I should load an LTO_section and read the information?
However, I'm still a bit confused since I believe input_cgraph gets called
before my pass. My pass is running during WPA. Besides, I can inspect the
initializers for other variables just not those of structs.
Any help will be appreciated. Thanks again!
>
>>
>>> Would people be interested in me extending this API to print
>>> out something more useful? I already have some code that
>>> iterates over fields and prints out names and types. It
>>> can still use some work though.
>>>
>>> However, I understand if this is the intended behaviour.
>>> Another question (related to the possibility of intended behaviour)
>>> is: how is struct initialization represented in trees?
>>> (I.e. will each field be initialized separately or something else
>>> happens?)
>>>
>>> Thanks! I also include the patch in case people want a simple
>>> working example on how to print declarations.
>>>
>>> diff --git a/gcc/Makefile.in b/gcc/Makefile.in
>>> index 6b857bd..f4f1376 100644
>>> --- a/gcc/Makefile.in
>>> +++ b/gcc/Makefile.in
>>> @@ -1368,6 +1368,7 @@ OBJS = \
>>> incpath.o \
>>> init-regs.o \
>>> internal-fn.o \
>>> + ipa-hello-world.o \
>>> ipa-cp.o \
>>> ipa-sra.o \
>>> ipa-devirt.o \
>>> diff --git a/gcc/common.opt b/gcc/common.opt
>>> index b4dc31c..304ec9f 100644
>>> --- a/gcc/common.opt
>>> +++ b/gcc/common.opt
>>> @@ -3348,4 +3348,7 @@ fipa-ra
>>> Common Report Var(flag_ipa_ra) Optimization
>>> Use caller save register across calls if possible.
>>>
>>> +fipa-typelist
>>> +Common Report Var(flag_ipa_typelist) TBD
>>> +
>>> ; This comment is to ensure we retain the blank line above.
>>> diff --git a/gcc/ipa-hello-world.c b/gcc/ipa-hello-world.c
>>> new file mode 100644
>>> index 000..ed4ae11
>>> --- /dev/null
>>> +++ b/gcc/ipa-hello-world.c
Does gcc automatically lower optimization level for very large routines?
Hi,
When using GCC to compile a very large routine with -O2, it failed with out of
memory during run time. (O1 is Okay)
As I checked within gdb, when “cc1” was consuming around 95% of the memory,
it’s at :
(gdb) where
#0 0x00ddbcb3 in df_chain_create (src=0x631006480f08,
dst=0x63100f306288) at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2267
#1 0x001a in df_chain_create_bb_process_use (
local_rd=0x7ffc109bfaf0, use=0x63100f306288, top_flag=0)
at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2441
#2 0x00dde5a7 in df_chain_create_bb (bb_index=16413)
at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2490
#3 0x00ddeaa9 in df_chain_finalize (all_blocks=0x63100097ac28)
at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2519
#4 0x00dbe95e in df_analyze_problem (dflow=0x60600027f740,
blocks_to_consider=0x63100097ac28, postorder=0x7f23761f1800,
n_blocks=40768) at ../../gcc-8.2.1-20180905/gcc/df-core.c:1179
#5 0x00dbedac in df_analyze_1 ()
….
The routine that was compiled is very big, has about 119258 lines of code.
I suspected that GCC’s data flow analysis might not handle very large routine
very well, consume too much memory, therefore out of memory for very big
routines.
Currently, I found one GCC’s source level pragma,
#pragma GCC optimize ("O1”)
And added it before the large routine (also added another one #pragma GCC
reset_options after the routine), this workaround the out of memory issue for
now.
However, manually locating large routines is time consuming, I am wondering
whether GCC can automatically detect large routines and lower the optimization
for those
Routines automatically? Or is there any internal parameters inside GCC’s data
flow analysis that compute the complexity of the routine, if it’s very big,
then will turn off
The aggressive analysis automatically? Or any option provided to end user to
control the aggressive data flow manually ?
Thanks a lot for any help.
Qing
Re: Commit messages and the move to git
On Thu, 19 Dec 2019 at 16:33, Joseph Myers wrote: > > On Thu, 19 Dec 2019, Jonathan Wakely wrote: > > > On Thu, 19 Dec 2019 at 15:49, Joseph Myers wrote: > > > > > > On Thu, 19 Dec 2019, Richard Earnshaw (lists) wrote: > > > > > > > > Done. https://gitlab.com/esr/gcc-conversion/merge_requests/25 fixes > > > > > (most of?) the most egregious ones, like fortran fixes with c++ PR > > > > > numbers and vice versa. Jakub and I have several whitelist commits > > > > > too, but I think they're all less critical (as mentioned previously, > > > > > most of them are unimportant differences like rtl-optimization vs > > > > > tree-optimization, or debug vs lto, or simply that the Bugzilla > > > > > component has been changed since the commit was done and the commit > > > > > was actually right at the time). > > > > > > > > > > > > > Merged > > > > > > And I've fixed up some Python syntax there (commas that needed to be > > > colons, between the commit number and the fixup information). > > > > Thanks. > > > > I'm afraid it was too mind-numbing to just work through the list from > > bottom to top. Jakub and I found some problems and some that can be > > whitelisted. I submitted another merge request. > > Thanks, merged. > > If people have done updates to components in Bugzilla and want a fresh > list generated using fresh Bugzilla data and the updates to component > aliases, I can readily do that (using the file of commit messages left > behind from my last conversion run), load on sourceware permitting. I've updated a tiny handful of bugs (just two IIRC) where the component was obviously suboptimal. Jakub and I came up with the following list of suggestions for component changes: 92324 from c to tree-optimization 91763 from go to lto 91772 from lto to debug 91137 from rtl-optimization to tree-optimization 91445 from c++ to tree-optimization 90577 from middle-end to fortran 90716 from debug to tree-optimization 90273 from debug to tree-optimization 90194 from debug to middle-end 89487 rtl-opt to tree-opt 45869 from c to tree-optimization 45176 from c to middle-end 45605 from c++ to tree-optimization 45838 from libgomp to middle-end 44536 from middle-end to fortran 41921 from c++ to lto 43611 from tree-optimization to c++ 47541 from c++ to tree-optimization 47293 from fortran to libquadmath 41094 from c++ to middle-end 40492 from c++ to tree-optimization
Re: Commit messages and the move to git
On Thu, 19 Dec 2019, Jonathan Wakely wrote: > Jakub and I came up with the following list of suggestions for > component changes: Since we don't normally review changes to individual bugs, if you think the new component is better than the old one (is a better representation of the subject area of the bug, not just more convenient for bugdb.py) I think you should just go ahead and make the change. -- Joseph S. Myers jos...@codesourcery.com
Re: Could I obtain the forms needed to make a contribution?
Eric Curtin writes: > I want to add a compiler warning, if it will get accepted. It's a > -Wlines warning. My employer bans the __LINE__ macro as well as the > ones warned by the -Wdate-time warning, because there is a consensus > that the addition of whitespace or comments should not yield different > binary output for our project. Would this patch get accepted? Sounds like a useful feature to me FWIW. Perhaps we should have a general warning that takes macro names as arguments, with __LINE__ being just one possibility. That might be over-designing it though. :-) Richard
Re: Could I obtain the forms needed to make a contribution?
Why not just add "-D__LINE__=LinkerError_LineMacroUsed_DoNotDoThat()" to CFLAGS? Best Regards, Dmitry Grinberg On Mon, Dec 16, 2019 at 3:51 AM Eric Curtin wrote: > I want to add a compiler warning, if it will get accepted. It's a > -Wlines warning. My employer bans the __LINE__ macro as well as the > ones warned by the -Wdate-time warning, because there is a consensus > that the addition of whitespace or comments should not yield different > binary output for our project. Would this patch get accepted? >
How to run LTO tests?
Hi,
I'm looking to create new tests for an LTO pass that I'm working on.
So, I started by trying to run the tests under the folder:
$gcc/gcc/testsuite/gcc.dg/lto
Looking at the documentation available here:
https://gcc.gnu.org/install/test.html
It says the following
In order to run sets of tests selectively, there are targets
[...] ‘make check-lto’ in the gcc subdirectory of the object directory.
And so, I go to gcc subdirectory of the object directory and
type make check-lto. However, I get this error message:
make: Nothing to be done for `check-lto'.
I have looked into the Makefile, and there is indeed a check-lto
target but make outputs "Nothing to be done for `check-lto'."
Just to be complete, these are the configure flags I used
when installing gcc:
../configure \
--disable-bootstrap \
--disable-libsanitizer \
--enable-__cxa_atexit \
--enable-shared \
--disable-libsanitizer \
--enable-languages=c,c++,fortran \
--enable-lto \
--enable-gold \
--enable-linker-build-id \
--with-cpu-emag \
--prefix="$HOME/code/${installdir}/"
Can anyone tell me how can I run the lto tests?
Thanks!
Re: How to run LTO tests?
On Thu, Dec 19, 2019 at 12:48 PM Erick Ochoa
wrote:
>
> Hi,
>
> I'm looking to create new tests for an LTO pass that I'm working on.
> So, I started by trying to run the tests under the folder:
> $gcc/gcc/testsuite/gcc.dg/lto
>
> Looking at the documentation available here:
> https://gcc.gnu.org/install/test.html
>
> It says the following
>
> In order to run sets of tests selectively, there are targets
> [...] ‘make check-lto’ in the gcc subdirectory of the object directory.
>
> And so, I go to gcc subdirectory of the object directory and
> type make check-lto. However, I get this error message:
>
> make: Nothing to be done for `check-lto'.
>
> I have looked into the Makefile, and there is indeed a check-lto
> target but make outputs "Nothing to be done for `check-lto'."
>
> Just to be complete, these are the configure flags I used
> when installing gcc:
>
> ../configure \
> --disable-bootstrap \
> --disable-libsanitizer \
> --enable-__cxa_atexit \
> --enable-shared \
> --disable-libsanitizer \
> --enable-languages=c,c++,fortran \
> --enable-lto \
> --enable-gold \
> --enable-linker-build-id \
> --with-cpu-emag \
> --prefix="$HOME/code/${installdir}/"
>
> Can anyone tell me how can I run the lto tests?
Did you try:
make check-gcc RUNTESTFLAGS="lto.exp" ?
Thanks,
Andrew Pinski
>
> Thanks!
Re: How to run LTO tests?
On 2019-12-19 3:50 p.m., Andrew Pinski wrote:
> On Thu, Dec 19, 2019 at 12:48 PM Erick Ochoa
> wrote:
>>
>> Hi,
>>
>> I'm looking to create new tests for an LTO pass that I'm working on.
>> So, I started by trying to run the tests under the folder:
>> $gcc/gcc/testsuite/gcc.dg/lto
>>
>> Looking at the documentation available here:
>> https://gcc.gnu.org/install/test.html
>>
>> It says the following
>>
>> In order to run sets of tests selectively, there are targets
>> [...] ‘make check-lto’ in the gcc subdirectory of the object directory.
>>
>> And so, I go to gcc subdirectory of the object directory and
>> type make check-lto. However, I get this error message:
>>
>> make: Nothing to be done for `check-lto'.
>>
>> I have looked into the Makefile, and there is indeed a check-lto
>> target but make outputs "Nothing to be done for `check-lto'."
>>
>> Just to be complete, these are the configure flags I used
>> when installing gcc:
>>
>> ../configure \
>> --disable-bootstrap \
>> --disable-libsanitizer \
>> --enable-__cxa_atexit \
>> --enable-shared \
>> --disable-libsanitizer \
>> --enable-languages=c,c++,fortran \
>> --enable-lto \
>> --enable-gold \
>> --enable-linker-build-id \
>> --with-cpu-emag \
>> --prefix="$HOME/code/${installdir}/"
>>
>> Can anyone tell me how can I run the lto tests?
>
> Did you try:
> make check-gcc RUNTESTFLAGS="lto.exp" ?
This works! Thanks!
>
> Thanks,
> Andrew Pinski
>
>>
>> Thanks!
Option flag with string arguments
Hello,
I am working on testing an optimization. I am starting to write
tests in the GCC testing suite. However, I want to develop some
fine grain testing for my own sake.
This optimization I am working on, is a variant of struct reordering.
One way I would like to test my pass is for example, making a test case
and guaranteeing that my pass is able to detect global variables of
a specific struct type.
For example, given the following C file
```
/* { dg-lto-options {{-flto -fipa-struct-reorg
-fipa-struct-reorg-assert-has-structs=astruct_s}} } */
struct astruct_s { _Bool a; };
struct astruct_s astruct;
int main() { };
```
I would like to create the option flag that has a list of string arguments
-fipa-struct-reorg-assert-has-structs=+
such that during the analysis time, I'd be able to have an assertion
to make sure that my pass has actually collected the types identified by the
strings.
I'm not very familiar on the DSL to specify option flags.
I've looked at gcc/common.opt for some examples and found this one:
```
frandom-seed=
Common Joined RejectNegative Var(common_deferred_options) Defer
-frandom-seed= Make compile reproducible using .
```
Could anyone explain how to specify my flag? Or possibly
point to some documentation/source that explains Common, Joined, etc...?
Re: Option flag with string arguments
On Thu, 19 Dec 2019 16:47:42 -0500
Erick Ochoa wrote:
> Hello,
>
> I am working on testing an optimization. I am starting to write
> tests in the GCC testing suite. However, I want to develop some
> fine grain testing for my own sake.
>
> This optimization I am working on, is a variant of struct reordering.
> One way I would like to test my pass is for example, making a test case
> and guaranteeing that my pass is able to detect global variables of
> a specific struct type.
>
> For example, given the following C file
>
> ```
> /* { dg-lto-options {{-flto -fipa-struct-reorg
> -fipa-struct-reorg-assert-has-structs=astruct_s}} } */
>
> struct astruct_s { _Bool a; };
> struct astruct_s astruct;
>
> int main() { };
> ```
>
> I would like to create the option flag that has a list of string arguments
>
> -fipa-struct-reorg-assert-has-structs=+
>
> such that during the analysis time, I'd be able to have an assertion
> to make sure that my pass has actually collected the types identified by the
> strings.
> I'm not very familiar on the DSL to specify option flags.
> I've looked at gcc/common.opt for some examples and found this one:
>
> ```
> frandom-seed=
> Common Joined RejectNegative Var(common_deferred_options) Defer
> -frandom-seed= Make compile reproducible using .
> ```
>
> Could anyone explain how to specify my flag? Or possibly
> point to some documentation/source that explains Common, Joined, etc...?
Have you looked at the GCC Internals Manual (gccint)? The section on option
specification files is online here:
https://gcc.gnu.org/onlinedocs/gccint/Option-properties.html#Option-properties
Jozef
patch to fix PR92905
The following patch fixes
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92905
The patch was successfully bootstrapped on x86-64 and ppc64 and
benchmarked on SPEC2000 on x86-64.
Committed as r279596
Index: ChangeLog
===
--- ChangeLog (revision 279595)
+++ ChangeLog (working copy)
@@ -1,3 +1,9 @@
+2019-12-19 Vladimir Makarov
+
+ PR target/92905
+ * lra-constraints.c (process_alt_operands): Check offmemok when
+ processing preferred_reload_class.
+
2019-12-19 Andrew Stubbs
* config/gcn/gcn-valu.md
Index: lra-constraints.c
===
--- lra-constraints.c (revision 279523)
+++ lra-constraints.c (working copy)
@@ -2722,11 +2722,24 @@ process_alt_operands (int only_alternati
&& (targetm.preferred_output_reload_class
(op, this_alternative) == NO_REGS
{
- if (lra_dump_file != NULL)
- fprintf (lra_dump_file,
- "%d Non-prefered reload: reject+=%d\n",
- nop, LRA_MAX_REJECT);
- reject += LRA_MAX_REJECT;
+ if (offmemok && REG_P (op))
+ {
+ if (lra_dump_file != NULL)
+ fprintf
+ (lra_dump_file,
+ "%d Spill pseudo into memory: reject+=3\n",
+ nop);
+ reject += 3;
+ }
+ else
+ {
+ if (lra_dump_file != NULL)
+ fprintf
+ (lra_dump_file,
+ "%d Non-prefered reload: reject+=%d\n",
+ nop, LRA_MAX_REJECT);
+ reject += LRA_MAX_REJECT;
+ }
}
if (! (MEM_P (op) && offmemok)
Index: testsuite/ChangeLog
===
--- testsuite/ChangeLog (revision 279595)
+++ testsuite/ChangeLog (working copy)
@@ -1,3 +1,8 @@
+2019-12-19 Vladimir Makarov
+
+ PR target/92905
+ * gcc.target/i386/pr92905.c: New test.
+
2019-12-19 Richard Sandiford
* g++.dg/ext/sve-sizeless-2.C: Don't expect an error for
Index: testsuite/gcc.target/i386/pr92905.c
===
--- testsuite/gcc.target/i386/pr92905.c (nonexistent)
+++ testsuite/gcc.target/i386/pr92905.c (working copy)
@@ -0,0 +1,9 @@
+/* { dg-do compile { target { lp64 } } } */
+/* { dg-options "-O2 -mtune=intel" } */
+float f(float x)
+{
+union {float f; unsigned i;} u = {x};
+u.i |= 0x8000;
+return u.f;
+}
+/* { dg-final { scan-assembler-not "rsp"} } */
Re: Option flag with string arguments
On 2019-12-19 5:01 p.m., Jozef Lawrynowicz wrote:
> On Thu, 19 Dec 2019 16:47:42 -0500
> Erick Ochoa wrote:
>
>> Hello,
>>
>> I am working on testing an optimization. I am starting to write
>> tests in the GCC testing suite. However, I want to develop some
>> fine grain testing for my own sake.
>>
>> This optimization I am working on, is a variant of struct reordering.
>> One way I would like to test my pass is for example, making a test case
>> and guaranteeing that my pass is able to detect global variables of
>> a specific struct type.
>>
>> For example, given the following C file
>>
>> ```
>> /* { dg-lto-options {{-flto -fipa-struct-reorg
>> -fipa-struct-reorg-assert-has-structs=astruct_s}} } */
>>
>> struct astruct_s { _Bool a; };
>> struct astruct_s astruct;
>>
>> int main() { };
>> ```
>>
>> I would like to create the option flag that has a list of string arguments
>>
>> -fipa-struct-reorg-assert-has-structs=+
>>
>> such that during the analysis time, I'd be able to have an assertion
>> to make sure that my pass has actually collected the types identified by the
>> strings.
>> I'm not very familiar on the DSL to specify option flags.
>> I've looked at gcc/common.opt for some examples and found this one:
>>
>> ```
>> frandom-seed=
>> Common Joined RejectNegative Var(common_deferred_options) Defer
>> -frandom-seed= Make compile reproducible using .
>> ```
>>
>> Could anyone explain how to specify my flag? Or possibly
>> point to some documentation/source that explains Common, Joined, etc...?
>
> Have you looked at the GCC Internals Manual (gccint)? The section on option
> specification files is online here:
> https://gcc.gnu.org/onlinedocs/gccint/Option-properties.html#Option-properties>
>
> Jozef
Thanks Jozef! This is exactly what I was looking for (and overlooked).
>
Re: Option flag with string arguments
On Thu, 2019-12-19 at 16:47 -0500, Erick Ochoa wrote:
> Hello,
>
> I am working on testing an optimization. I am starting to write
> tests in the GCC testing suite. However, I want to develop some
> fine grain testing for my own sake.
>
> This optimization I am working on, is a variant of struct reordering.
> One way I would like to test my pass is for example, making a test
> case
> and guaranteeing that my pass is able to detect global variables of
> a specific struct type.
For example, given the following C file
>
> ```
> /* { dg-lto-options {{-flto -fipa-struct-reorg -fipa-struct-reorg-
> assert-has-structs=astruct_s}} } */
>
> struct astruct_s { _Bool a; };
> struct astruct_s astruct;
>
> int main() { };
> ```
>
> I would like to create the option flag that has a list of string
> arguments
>
> -fipa-struct-reorg-assert-has-structs=+
>
> such that during the analysis time, I'd be able to have an assertion
> to make sure that my pass has actually collected the types identified
> by the
> strings.
> I'm not very familiar on the DSL to specify option flags.
> I've looked at gcc/common.opt for some examples and found this one:
>
> ```
> frandom-seed=
> Common Joined RejectNegative Var(common_deferred_options) Defer
> -frandom-seed= Make compile reproducible using .
> ```
>
> Could anyone explain how to specify my flag? Or possibly
> point to some documentation/source that explains Common, Joined,
> etc...?
It sounds like what you really want is a DejaGnu option to detect that
a specific struct type was handled by your pass, rather than a command-
line option for controlling that pass.
I think you want something like:
/* { dg-options "-fdump-ipa-your-pass" } */
/* { dg-lto-options {{-flto -fipa-struct-reorg }} } */
(not sure of the precise interaction of these two though)
and:
/* { dg-final { scan-ipa-dump "handling type: 'struct_s'" "your-pass" } } */
then arranging for the dumpfile for your-pass to contain the message:
dg-final queues up a scan of the given dumpfile which will PASS/FAIL
based on whether it sees the message.
Try grepping the testsuite for dg-final to see more examples of the
idea.
Hope this is helpful
Dave
Re: Does gcc automatically lower optimization level for very large routines?
This issue is well-known in research/scientific software. The problem of
compiler hang or RAM overconsumption is actually not about the routine
size, but about too complicated control flow. When optimizing, the compiler
traverses the control flow graph, which may have the misfortune to explode
in terms of complexity. So you may want to check whether your routine
heavily deploys nested cascades of "if ... else" or goto-s. That is, the
routine size is not a good metric to catch this behavior. GCC may rather
attempt "reversible" strategy of optimizations to stop and undo those that
get beyond a certain threshold.
Kind regards,
- Dmitry.
чт, 19 дек. 2019 г. в 17:38, Qing Zhao :
> Hi,
>
> When using GCC to compile a very large routine with -O2, it failed with
> out of memory during run time. (O1 is Okay)
>
> As I checked within gdb, when “cc1” was consuming around 95% of the
> memory, it’s at :
>
> (gdb) where
> #0 0x00ddbcb3 in df_chain_create (src=0x631006480f08,
> dst=0x63100f306288) at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2267
> #1 0x001a in df_chain_create_bb_process_use (
> local_rd=0x7ffc109bfaf0, use=0x63100f306288, top_flag=0)
> at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2441
> #2 0x00dde5a7 in df_chain_create_bb (bb_index=16413)
> at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2490
> #3 0x00ddeaa9 in df_chain_finalize (all_blocks=0x63100097ac28)
> at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2519
> #4 0x00dbe95e in df_analyze_problem (dflow=0x60600027f740,
> blocks_to_consider=0x63100097ac28, postorder=0x7f23761f1800,
> n_blocks=40768) at ../../gcc-8.2.1-20180905/gcc/df-core.c:1179
> #5 0x00dbedac in df_analyze_1 ()
> ….
>
> The routine that was compiled is very big, has about 119258 lines of code.
> I suspected that GCC’s data flow analysis might not handle very large
> routine very well, consume too much memory, therefore out of memory for
> very big routines.
>
> Currently, I found one GCC’s source level pragma,
>
> #pragma GCC optimize ("O1”)
>
> And added it before the large routine (also added another one #pragma GCC
> reset_options after the routine), this workaround the out of memory issue
> for now.
>
> However, manually locating large routines is time consuming, I am
> wondering whether GCC can automatically detect large routines and lower the
> optimization for those
> Routines automatically? Or is there any internal parameters inside GCC’s
> data flow analysis that compute the complexity of the routine, if it’s very
> big, then will turn off
> The aggressive analysis automatically? Or any option provided to end user
> to control the aggressive data flow manually ?
>
>
> Thanks a lot for any help.
>
> Qing
Re: Does gcc automatically lower optimization level for very large routines?
Hi, Dmitry,
Thanks for the responds.
Yes, routine size only cannot determine the complexity of the routine.
Different compiler analysis might have different formula with multiple
parameters to compute its complexity.
However, the common issue is: when the complexity of a specific routine for a
specific compiler analysis exceeds a threshold, the compiler might consume all
the available memory and abort the compilation.
Therefore, in order to avoid the failed compilation due to out of memory, some
compilers might set a threshold for the complexity of a specific compiler
analysis (for example, the more aggressive data flow analysis), when the
threshold is met, the specific aggressive analysis will be turned off for this
specific routine. Or the optimization level will be lowered for the specific
routine (and given a warning during compilation time for such adjustment).
I am wondering whether GCC has such capability? Or any option provided to
increase or decrease the threshold for some of the common analysis (for
example, data flow)?
Thanks.
Qing
> On Dec 19, 2019, at 4:50 PM, Dmitry Mikushin wrote:
>
> This issue is well-known in research/scientific software. The problem of
> compiler hang or RAM overconsumption is actually not about the routine
> size, but about too complicated control flow. When optimizing, the compiler
> traverses the control flow graph, which may have the misfortune to explode
> in terms of complexity. So you may want to check whether your routine
> heavily deploys nested cascades of "if ... else" or goto-s. That is, the
> routine size is not a good metric to catch this behavior. GCC may rather
> attempt "reversible" strategy of optimizations to stop and undo those that
> get beyond a certain threshold.
>
> Kind regards,
> - Dmitry.
>
>
> чт, 19 дек. 2019 г. в 17:38, Qing Zhao :
>
>> Hi,
>>
>> When using GCC to compile a very large routine with -O2, it failed with
>> out of memory during run time. (O1 is Okay)
>>
>> As I checked within gdb, when “cc1” was consuming around 95% of the
>> memory, it’s at :
>>
>> (gdb) where
>> #0 0x00ddbcb3 in df_chain_create (src=0x631006480f08,
>>dst=0x63100f306288) at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2267
>> #1 0x001a in df_chain_create_bb_process_use (
>>local_rd=0x7ffc109bfaf0, use=0x63100f306288, top_flag=0)
>>at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2441
>> #2 0x00dde5a7 in df_chain_create_bb (bb_index=16413)
>>at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2490
>> #3 0x00ddeaa9 in df_chain_finalize (all_blocks=0x63100097ac28)
>>at ../../gcc-8.2.1-20180905/gcc/df-problems.c:2519
>> #4 0x00dbe95e in df_analyze_problem (dflow=0x60600027f740,
>>blocks_to_consider=0x63100097ac28, postorder=0x7f23761f1800,
>>n_blocks=40768) at ../../gcc-8.2.1-20180905/gcc/df-core.c:1179
>> #5 0x00dbedac in df_analyze_1 ()
>> ….
>>
>> The routine that was compiled is very big, has about 119258 lines of code.
>> I suspected that GCC’s data flow analysis might not handle very large
>> routine very well, consume too much memory, therefore out of memory for
>> very big routines.
>>
>> Currently, I found one GCC’s source level pragma,
>>
>> #pragma GCC optimize ("O1”)
>>
>> And added it before the large routine (also added another one #pragma GCC
>> reset_options after the routine), this workaround the out of memory issue
>> for now.
>>
>> However, manually locating large routines is time consuming, I am
>> wondering whether GCC can automatically detect large routines and lower the
>> optimization for those
>> Routines automatically? Or is there any internal parameters inside GCC’s
>> data flow analysis that compute the complexity of the routine, if it’s very
>> big, then will turn off
>> The aggressive analysis automatically? Or any option provided to end user
>> to control the aggressive data flow manually ?
>>
>>
>> Thanks a lot for any help.
>>
>> Qing
Re: Errors building libgcc for powerpc64le-linux-gnu
On Wed, Dec 18, 2019 at 7:58 AM Segher Boessenkool wrote: > > On Sun, Dec 15, 2019 at 09:43:20AM -0800, Ian Lance Taylor wrote: > > On Sat, Dec 14, 2019 at 11:25 PM Segher Boessenkool > > wrote: > > > > > > On Sat, Dec 14, 2019 at 10:51:50AM -0800, Ian Lance Taylor via gcc wrote: > > > > I'm seeing compiler crashes building libgcc for powerpc64le-linux-gnu, > > > > cross-compiling from x86_64-pc-linux-gnu. I'm at SVN revision 279830. > > > > I'm seeing the following. Is anybody else seeing this crash? Thanks. > > > > > > No, and that makes me wonder what is going on. The error is simple enough > > > of course, as you note in a later message; but why do we not see it on > > > every other build? > > > > I think it's because clang treats a left shift by a negative number as > > undefined behavior but GCC does not. So GCC is consistently producing > > some number, and clang is producing different numbers. > > > > I should note that I don't really understand what purpose that > > constant is serving anyhow. > > It's the (old, changed months ago!) register number for CR7. This code > creates the bitfield that says which CR fields to save or restore. That much I got, but the bitfield doesn't seem to actually be used for anything. It seems to be created and matched but not used. Unless I missed something. The constant doesn't seem to add anything we don't already know from the register number. > Anyway, should be fixed now. If you can confirm, please do :-) Confirmed. Thanks. Ian
Re: Errors building libgcc for powerpc64le-linux-gnu
On Thu, Dec 19, 2019 at 04:08:39PM -0800, Ian Lance Taylor wrote:
> On Wed, Dec 18, 2019 at 7:58 AM Segher Boessenkool
> wrote:
> >
> > On Sun, Dec 15, 2019 at 09:43:20AM -0800, Ian Lance Taylor wrote:
> > > On Sat, Dec 14, 2019 at 11:25 PM Segher Boessenkool
> > > wrote:
> > > >
> > > > On Sat, Dec 14, 2019 at 10:51:50AM -0800, Ian Lance Taylor via gcc
> > > > wrote:
> > > I should note that I don't really understand what purpose that
> > > constant is serving anyhow.
> >
> > It's the (old, changed months ago!) register number for CR7. This code
> > creates the bitfield that says which CR fields to save or restore.
>
> That much I got, but the bitfield doesn't seem to actually be used for
> anything. It seems to be created and matched but not used. Unless I
> missed something. The constant doesn't seem to add anything we don't
> already know from the register number.
Ah okay, in that sense. In *movsi_to_cr any combination of bits can be
set. We still could derive the mask from the rtl vector every time it
is used, of course.
We use the mtocrf insn most of the time these days. ("o" means "one",
exactly one of the eight bits is set, it works on just one cr field).
If we had a different unspec for that (which is probably a good idea
anyway, thanks for that :-) ) we could easily do as you suggest.
> > Anyway, should be fixed now. If you can confirm, please do :-)
>
> Confirmed. Thanks.
Cheers!
Segher
Re: Does gcc automatically lower optimization level for very large routines?
On Thu, 2019-12-19 at 17:06 -0600, Qing Zhao wrote: > Hi, Dmitry, > > Thanks for the responds. > > Yes, routine size only cannot determine the complexity of the routine. > Different compiler analysis might have different formula with multiple > parameters to compute its complexity. > > However, the common issue is: when the complexity of a specific routine for a > specific compiler analysis exceeds a threshold, the compiler might consume > all the available memory and abort the compilation. > > Therefore, in order to avoid the failed compilation due to out of memory, > some compilers might set a threshold for the complexity of a specific > compiler analysis (for example, the more aggressive data flow analysis), when > the threshold is met, the specific aggressive analysis will be turned off for > this specific routine. Or the optimization level will be lowered for the > specific routine (and given a warning during compilation time for such > adjustment). > > I am wondering whether GCC has such capability? Or any option provided to > increase or decrease the threshold for some of the common analysis (for > example, data flow)? > There are various places where if we hit a limit, then we throttle optimization. But it's not done consistently or pervasively. Those limits are typically around things like CFG complexity. We do _not_ try to recover after an out of memory error, or anything like that. jeff >
Re: Does gcc automatically lower optimization level for very large routines?
On Thu, Dec 19, 2019 at 7:41 PM Jeff Law wrote: > > On Thu, 2019-12-19 at 17:06 -0600, Qing Zhao wrote: > > Hi, Dmitry, > > > > Thanks for the responds. > > > > Yes, routine size only cannot determine the complexity of the routine. > > Different compiler analysis might have different formula with multiple > > parameters to compute its complexity. > > > > However, the common issue is: when the complexity of a specific routine for > > a specific compiler analysis exceeds a threshold, the compiler might > > consume all the available memory and abort the compilation. > > > > Therefore, in order to avoid the failed compilation due to out of memory, > > some compilers might set a threshold for the complexity of a specific > > compiler analysis (for example, the more aggressive data flow analysis), > > when the threshold is met, the specific aggressive analysis will be turned > > off for this specific routine. Or the optimization level will be lowered > > for the specific routine (and given a warning during compilation time for > > such adjustment). > > > > I am wondering whether GCC has such capability? Or any option provided to > > increase or decrease the threshold for some of the common analysis (for > > example, data flow)? > > > There are various places where if we hit a limit, then we throttle > optimization. But it's not done consistently or pervasively. > > Those limits are typically around things like CFG complexity. > > We do _not_ try to recover after an out of memory error, or anything > like that. I have mentioned a few times before that IBM XL Compiler allows the user to specify the maximum memory utilization for the compiler (including "unlimmited"). The compiler optimization passes estimate the memory usage for the data structures of each optimization pass. The the memory usage is too high, the pass attempts to sub-divide the region and calculates the estimated memory usage again, recursing until it can apply the optimization within the memory limit or the optimization would not be effective. IBM XL Compiler does not try to recover from an out of memory error, but it explicitly considers memory use of optimization passes. It does not adjust the complexity of the optimization, but it does adjust the size of the region or other parameters to reduce the memory usage of the data structures for an optimization. Thanks, David
Re: Does gcc automatically lower optimization level for very large routines?
Trying to plan memory consumption ahead-of-work contradicts with the nature of the graph traversal. Estimation may work very well for something simple like linear or log-linear behavior. But many compiler algorithms are known to be polynomial or exponential (or even worse in case of bugs). So, estimation is a nice step ahead, but only fallback & recover can ultimately solve the problem. пт, 20 дек. 2019 г. в 02:32, David Edelsohn : > On Thu, Dec 19, 2019 at 7:41 PM Jeff Law wrote: > > > > On Thu, 2019-12-19 at 17:06 -0600, Qing Zhao wrote: > > > Hi, Dmitry, > > > > > > Thanks for the responds. > > > > > > Yes, routine size only cannot determine the complexity of the routine. > Different compiler analysis might have different formula with multiple > parameters to compute its complexity. > > > > > > However, the common issue is: when the complexity of a specific > routine for a specific compiler analysis exceeds a threshold, the compiler > might consume all the available memory and abort the compilation. > > > > > > Therefore, in order to avoid the failed compilation due to out of > memory, some compilers might set a threshold for the complexity of a > specific compiler analysis (for example, the more aggressive data flow > analysis), when the threshold is met, the specific aggressive analysis will > be turned off for this specific routine. Or the optimization level will be > lowered for the specific routine (and given a warning during compilation > time for such adjustment). > > > > > > I am wondering whether GCC has such capability? Or any option provided > to increase or decrease the threshold for some of the common analysis (for > example, data flow)? > > > > > There are various places where if we hit a limit, then we throttle > > optimization. But it's not done consistently or pervasively. > > > > Those limits are typically around things like CFG complexity. > > > > We do _not_ try to recover after an out of memory error, or anything > > like that. > > I have mentioned a few times before that IBM XL Compiler allows the > user to specify the maximum memory utilization for the compiler > (including "unlimmited"). The compiler optimization passes estimate > the memory usage for the data structures of each optimization pass. > The the memory usage is too high, the pass attempts to sub-divide the > region and calculates the estimated memory usage again, recursing > until it can apply the optimization within the memory limit or the > optimization would not be effective. IBM XL Compiler does not try to > recover from an out of memory error, but it explicitly considers > memory use of optimization passes. It does not adjust the complexity > of the optimization, but it does adjust the size of the region or > other parameters to reduce the memory usage of the data structures for > an optimization. > > Thanks, David >
Need sanity check on DSE vs expander issue
I need a sanity check here.
Given this code:
> typedef union { long double value; unsigned int word[4]; } memory_long_double;
> static unsigned int ored_words[4];
> static void add_to_ored_words (long double x)
> {
> memory_long_double m;
> size_t i;
> memset (&m, 0, sizeof (m));
> m.value = x;
> for (i = 0; i < 4; i++)
> {
> ored_words[i] |= m.word[i];
> }
> }
>
DSE is removing the memset as it thinks the assignment to m.value is
going to set the entire union.
But when we translate that into RTL we use XFmode:
> ;; m.value ={v} x_6(D);
>
> (insn 7 6 0 (set (mem/v/j/c:XF (plus:DI (reg/f:DI 77 virtual-stack-vars)
> (const_int -16 [0xfff0])) [2 m.value+0 S16 A128])
> (reg/v:XF 86 [ x ])) "j.c":13:11 -1
> (nil))
>
That (of course) only writes 80 bits of data because of XFmode, leaving
48 bits uninitialized. We then read those bits, or-ing the
uninitialized data into ored_words and all hell breaks loose later.
Am I losing my mind? ISTM that dse and the expander have to agree on
how much data is written by the store to m.value.
Jeff
Re: Need sanity check on DSE vs expander issue
On December 20, 2019 3:20:40 AM GMT+01:00, Jeff Law wrote:
>I need a sanity check here.
>
>Given this code:
>
>> typedef union { long double value; unsigned int word[4]; }
>memory_long_double;
>> static unsigned int ored_words[4];
>> static void add_to_ored_words (long double x)
>> {
>> memory_long_double m;
>> size_t i;
>> memset (&m, 0, sizeof (m));
>> m.value = x;
>> for (i = 0; i < 4; i++)
>> {
>> ored_words[i] |= m.word[i];
>> }
>> }
>>
>
>DSE is removing the memset as it thinks the assignment to m.value is
>going to set the entire union.
>
>But when we translate that into RTL we use XFmode:
>
>> ;; m.value ={v} x_6(D);
>>
>> (insn 7 6 0 (set (mem/v/j/c:XF (plus:DI (reg/f:DI 77
>virtual-stack-vars)
>> (const_int -16 [0xfff0])) [2 m.value+0
>S16 A128])
>> (reg/v:XF 86 [ x ])) "j.c":13:11 -1
>> (nil))
>>
>
>That (of course) only writes 80 bits of data because of XFmode, leaving
>48 bits uninitialized. We then read those bits, or-ing the
>uninitialized data into ored_words and all hell breaks loose later.
>
>Am I losing my mind? ISTM that dse and the expander have to agree on
>how much data is written by the store to m.value.
It looks like MEM_SIZE is wrong here, so you need to figure how we arrive at
this (I guess TYPE_SIZE vs. MODE_SIZE mismatch is biting us here?)
That is, either the MEM should have BLKmode or the mode size should match
MEM_SIZE. Maybe DSE can avoid looking at MEM_SIZE for non-BLKmode MEMs?
Richard.
>
>Jeff
Re: Need sanity check on DSE vs expander issue
On Fri, 2019-12-20 at 08:09 +0100, Richard Biener wrote:
> On December 20, 2019 3:20:40 AM GMT+01:00, Jeff Law wrote:
> > I need a sanity check here.
> >
> > Given this code:
> >
> > > typedef union { long double value; unsigned int word[4]; }
> > memory_long_double;
> > > static unsigned int ored_words[4];
> > > static void add_to_ored_words (long double x)
> > > {
> > > memory_long_double m;
> > > size_t i;
> > > memset (&m, 0, sizeof (m));
> > > m.value = x;
> > > for (i = 0; i < 4; i++)
> > > {
> > > ored_words[i] |= m.word[i];
> > > }
> > > }
> > >
> >
> > DSE is removing the memset as it thinks the assignment to m.value is
> > going to set the entire union.
> >
> > But when we translate that into RTL we use XFmode:
> >
> > > ;; m.value ={v} x_6(D);
> > >
> > > (insn 7 6 0 (set (mem/v/j/c:XF (plus:DI (reg/f:DI 77
> > virtual-stack-vars)
> > > (const_int -16 [0xfff0])) [2 m.value+0
> > S16 A128])
> > > (reg/v:XF 86 [ x ])) "j.c":13:11 -1
> > > (nil))
> > >
> >
> > That (of course) only writes 80 bits of data because of XFmode, leaving
> > 48 bits uninitialized. We then read those bits, or-ing the
> > uninitialized data into ored_words and all hell breaks loose later.
> >
> > Am I losing my mind? ISTM that dse and the expander have to agree on
> > how much data is written by the store to m.value.
>
> It looks like MEM_SIZE is wrong here, so you need to figure how we arrive at
> this (I guess TYPE_SIZE vs. MODE_SIZE mismatch is biting us here?)
>
> That is, either the MEM should have BLKmode or the mode size should match
> MEM_SIZE. Maybe DSE can avoid looking at MEM_SIZE for non-BLKmode MEMs?
It's gimple DSE that removes the memset, so it shouldn't be mucking
around with modes at all. stmt_kills_ref_p seems to think the
assignment to m.value sets all of m.
The ao_ref for memset looks reasonable:
> (gdb) p *ref
> $14 = {ref = 0x0, base = 0x77ffbea0, offset = {> =
> {coeffs = {0}}, },
> size = {> = {coeffs = {128}}, },
> max_size = {> = {
> coeffs = {128}}, }, ref_alias_set = 0, base_alias_set =
> 0, volatile_p = false}
>
128 bits with a base of VAR_DECL m.
We looking to see if this statement will kill the ref:
> (gdb) p debug_gimple_stmt (stmt)
> # .MEM_8 = VDEF <.MEM_6>
> m.value ={v} x_7(D);
> $21 = void
> (gdb) p debug_tree (lhs)
> type size
> unit-size
> align:128 warn_if_not_align:0 symtab:0 alias-set -1 canonical-type
> 0x7fffea988690 precision:80>
> side-effects volatile
> arg:0 type volatile type_0 BLK size unit-size
>
> align:128 warn_if_not_align:0 symtab:0 alias-set -1
> canonical-type 0x7fffea988348 fields
> context
> pointer_to_this >
> side-effects addressable volatile used read BLK j.c:10:31 size
> unit-size
> align:128 warn_if_not_align:0 context add_to_ored_words>
> chain size_t>
> used unsigned read DI j.c:11:10
> size
> unit-size
> align:64 warn_if_not_align:0 context 0x7fffea97bd00 add_to_ored_words>>>
> arg:1 type unit-size
> align:128 warn_if_not_align:0 symtab:0 alias-set -1
> canonical-type 0x7fffea8133f0 precision:80
> pointer_to_this >
> XF j.c:6:29 size unit-size
>
> align:128 warn_if_not_align:0 offset_align 128
> offset
> bit-offset context
>
> chain
> TI j.c:6:49 size unit-size
>
> align:32 warn_if_not_align:0 offset_align 128 offset 0x7fffea7f3d08 0> bit-offset context
> >>
> j.c:13:4 start: j.c:13:3 finish: j.c:13:9>
> $22 = void
>
stmt_kills_ref_p calls get_ref_base_and_extent on that LHS object. THe
returned base is the same as ref->base. The returned offset is zero
with size/max_size of 128 bits. So according to
get_ref_base_and_extent the assignment is going to write 128 bits and
thus kills the memset.
One might argue that's where the problems start -- somewhere in
get_ref_base_and_extent.
I'm largely offline the next couple weeks...
I don't have any "real" failures I'm tracking because of this, but it
does cause some configure generated tests to give the wrong result.
Thankfully the inconsistency just doesn't matter for any of the
packages that are affected.
Jeff
Re: Need sanity check on DSE vs expander issue
On Thu, Dec 19, 2019 at 11:25 PM Jeff Law wrote:
>
> On Fri, 2019-12-20 at 08:09 +0100, Richard Biener wrote:
> > On December 20, 2019 3:20:40 AM GMT+01:00, Jeff Law wrote:
> > > I need a sanity check here.
> > >
> > > Given this code:
> > >
> > > > typedef union { long double value; unsigned int word[4]; }
> > > memory_long_double;
> > > > static unsigned int ored_words[4];
> > > > static void add_to_ored_words (long double x)
> > > > {
> > > > memory_long_double m;
> > > > size_t i;
> > > > memset (&m, 0, sizeof (m));
> > > > m.value = x;
> > > > for (i = 0; i < 4; i++)
> > > > {
> > > > ored_words[i] |= m.word[i];
> > > > }
> > > > }
> > > >
> > >
> > > DSE is removing the memset as it thinks the assignment to m.value is
> > > going to set the entire union.
> > >
> > > But when we translate that into RTL we use XFmode:
> > >
> > > > ;; m.value ={v} x_6(D);
> > > >
> > > > (insn 7 6 0 (set (mem/v/j/c:XF (plus:DI (reg/f:DI 77
> > > virtual-stack-vars)
> > > > (const_int -16 [0xfff0])) [2 m.value+0
> > > S16 A128])
> > > > (reg/v:XF 86 [ x ])) "j.c":13:11 -1
> > > > (nil))
> > > >
> > >
> > > That (of course) only writes 80 bits of data because of XFmode, leaving
> > > 48 bits uninitialized. We then read those bits, or-ing the
> > > uninitialized data into ored_words and all hell breaks loose later.
> > >
> > > Am I losing my mind? ISTM that dse and the expander have to agree on
> > > how much data is written by the store to m.value.
> >
> > It looks like MEM_SIZE is wrong here, so you need to figure how we arrive
> > at this (I guess TYPE_SIZE vs. MODE_SIZE mismatch is biting us here?)
> >
> > That is, either the MEM should have BLKmode or the mode size should match
> > MEM_SIZE. Maybe DSE can avoid looking at MEM_SIZE for non-BLKmode MEMs?
> It's gimple DSE that removes the memset, so it shouldn't be mucking
> around with modes at all. stmt_kills_ref_p seems to think the
> assignment to m.value sets all of m.
Shouldn't stmt_kills_ref_p be checking TYPE_PRECISION here?
Though x86_64 target might be the only one where MEM_SIZE/TYPE_SIZE is
not fully written to when TYPE_PRECISION!=TYPE_SIZE though.
I think I heard m68k writes the full 96bits.
Thanks,
Andrew Pinski
>
> The ao_ref for memset looks reasonable:
>
> > (gdb) p *ref
> > $14 = {ref = 0x0, base = 0x77ffbea0, offset = {>
> > = {coeffs = {0}}, },
> > size = {> = {coeffs = {128}}, },
> > max_size = {> = {
> > coeffs = {128}}, }, ref_alias_set = 0, base_alias_set
> > = 0, volatile_p = false}
> >
> 128 bits with a base of VAR_DECL m.
>
> We looking to see if this statement will kill the ref:
>
> > (gdb) p debug_gimple_stmt (stmt)
> > # .MEM_8 = VDEF <.MEM_6>
> > m.value ={v} x_7(D);
> > $21 = void
> > (gdb) p debug_tree (lhs)
> > > type > size
> > unit-size
> > align:128 warn_if_not_align:0 symtab:0 alias-set -1 canonical-type
> > 0x7fffea988690 precision:80>
> > side-effects volatile
> > arg:0 > type > volatile type_0 BLK size unit-size
> >
> > align:128 warn_if_not_align:0 symtab:0 alias-set -1
> > canonical-type 0x7fffea988348 fields
> > context
> > pointer_to_this >
> > side-effects addressable volatile used read BLK j.c:10:31 size
> > unit-size
> > align:128 warn_if_not_align:0 context > add_to_ored_words>
> > chain > size_t>
> > used unsigned read DI j.c:11:10
> > size
> > unit-size
> > align:64 warn_if_not_align:0 context > 0x7fffea97bd00 add_to_ored_words>>>
> > arg:1 > type > unit-size
> > align:128 warn_if_not_align:0 symtab:0 alias-set -1
> > canonical-type 0x7fffea8133f0 precision:80
> > pointer_to_this >
> > XF j.c:6:29 size unit-size
> >
> > align:128 warn_if_not_align:0 offset_align 128
> > offset
> > bit-offset context
> >
> > chain > 0x7fffea981f18>
> > TI j.c:6:49 size unit-size
> >
> > align:32 warn_if_not_align:0 offset_align 128 offset
> > bit-offset
> > context >>
> > j.c:13:4 start: j.c:13:3 finish: j.c:13:9>
> > $22 = void
> >
>
> stmt_kills_ref_p calls get_ref_base_and_extent on that LHS object. THe
> returned base is the same as ref->base. The returned offset is zero
> with size/max_size of 128 bits. So according to
> get_ref_base_and_extent the assignment is going to write 128 bits and
> thus kills the memset.
>
> One might argue that's where the problems start -- somewhere in
> get_ref_base_and_extent.
>
> I'm largely offline the next couple weeks...
>
> I don't have any "real" failures I'm tracking because of this, but it
> does cause some configure generated tests to give the wrong result.
> Thankfully the inconsistency just doesn't matter for any of the
> packages that are affected.
>
>
> Jeff
>
