Re: Proposal for the transition timetable for the move to GIT

[Mark Wielaard]

Hi Maxim,

On Fri, 2019-12-06 at 17:44 +0300, Maxim Kuvyrkov wrote:
> > On Sep 19, 2019, at 6:34 PM, Maxim Kuvyrkov  
> > wrote:
> > > On Sep 17, 2019, at 3:02 PM, Richard Earnshaw (lists) 
> > >  wrote:
> > > 
> > > Monday 16th December 2019 - cut off date for picking which git conversion 
> > > to use
> > > 
> > > Tuesday 31st December 2019 - SVN repo becomes read-only at end of stage 3.
> > > 
> > > Thursday 2nd January 2020 - (ie read-only + 2 days) new git repo comes on 
> > > line for live commits.
> > 
> > I have regenerated my primary version this week, and it's up at
> > https://git.linaro.org/people/maxim-kuvyrkov/gcc-pretty.git/ .
> > So far I have received only minor issue reports about it, and all known 
> > problems have been fixed.
> > I could use a bit more scrutiny :-).
> 
> I think now is a good time to give status update on the svn->git conversion I 
> maintain.
> See https://git.linaro.org/people/maxim-kuvyrkov/gcc-pretty.git/ .
> 
> 1. The conversion has all SVN live branches converted as branches under 
> refs/heads/* .
> 
> 2. The conversion has all SVN live tags converted as annotated tags under 
> refs/tags/* .
> 
> 3. If desired, it would be trivial to add all deleted / leaf SVN branches and 
> tags.
>They would be named as branches/my-deleted-branch@12345,
>where @12345 is the revision at which the branch was deleted.
>Branches created and deleted multiple times would have separate entries
>corresponding to delete revisions.
> 
> 4. Git committer and git author entries are very accurate
>(imo, better than reposurgeon's, but I'm biased).
>Developers' names and email addresses are mined from commit logs,
>changelogs and source code and have historically-accurately attributions
>to employer's email addresses.
> 
> 5. Since there is interest in reparenting branches to fix cvs2svn merge 
> issues,
>I've added this feature to my scripts as well (turned out to be trivial).
>I'll keep the original gcc-pretty.git repo intact and will upload the new 
> one at
>https://git.linaro.org/people/maxim-kuvyrkov/gcc-reparent.git/
>-- should be live by Monday.

Should we go with the gcc-reparent.git repo now?

Where exactly should it be installed under https://gcc.gnu.org/git/
Replacing the existing gcc.git will be confusing, but then how would we
name the repo that will become the main git gcc repo in 2 weeks?

Where are the tools/scripts that should be installed on gcc.gnu.org to
keep it up to date during the next 2 week transition period?

Thanks,

Mark

Re: Proposal for the transition timetable for the move to GIT

[Joseph Myers]

On Mon, 16 Dec 2019, Mark Wielaard wrote:

> Should we go with the gcc-reparent.git repo now?

I think we should go with the reposurgeon conversion, with all Richard's 
improvements to commit messages.  gcc-reparent.git has issues of its own; 
at least, checking the list of branches shows some branches are missing.  
So both conversions can still be considered works in progress.

However, we should also note that stage 3 is intended to last two months, 
ending with the move to git 
 
, and given that it 
didn't start at the start of November as anticipated in the originally 
proposed timetable, that implies corresponding updates to all the dates.  
By now, enough people are away until the new year that now isn't a good 
time for deciding things anyway.

-- 
Joseph S. Myers
jos...@codesourcery.com

Executable file

[lindorx]

I want to  know how to cpmpile the specified executable format with GCC. I use 
GCC on the Windows platform.But I want to compile the ELF format file.
In addition,why did the following information appear when compiling the ".c" 
file on Linux.But I can compile normally with MinGW-gcc.


 error: inconsistent operand constraints in an ‘asm’
  __asm__ __volatile__("mov %1,%%dx\n\t"


gcc version : gcc 6.3.0


Please excuse my poor English.
| |
lindorx
|
|
lind...@163.com
|
签名由网易邮箱大师定制

Could I obtain the forms needed to make a contribution?

[Eric Curtin]

I want to add a compiler warning, if it will get accepted. It's a
-Wlines warning. My employer bans the __LINE__ macro as well as the
ones warned by the -Wdate-time warning, because there is a consensus
that the addition of whitespace or comments should not yield different
binary output for our project. Would this patch get accepted?

Re: Proposal for the transition timetable for the move to GIT

[Mark Wielaard]

On Mon, 2019-12-16 at 11:29 +, Joseph Myers wrote:
> On Mon, 16 Dec 2019, Mark Wielaard wrote:
> 
> > Should we go with the gcc-reparent.git repo now?
> 
> I think we should go with the reposurgeon conversion, with all Richard's 
> improvements to commit messages.  gcc-reparent.git has issues of its own; 
> at least, checking the list of branches shows some branches are missing.  
> So both conversions can still be considered works in progress.

I thought we would pick the best available conversion today.
If we keep tweaking the conversions till they are "perfect" we probably
never reach that point.

> However, we should also note that stage 3 is intended to last two months, 
> ending with the move to git 
>  
> , and given that it 
> didn't start at the start of November as anticipated in the originally 
> proposed timetable, that implies corresponding updates to all the dates.  
> By now, enough people are away until the new year that now isn't a good 
> time for deciding things anyway.

The idea was to do it while most people were away to have the least
impact. The timeline https://gcc.gnu.org/wiki/GitConversion does say we
can slip for logistical reasons the read-only date (2019/12/31) by a
few days.

Do people really want to keep tweaking the conversions and postpone the
git switchover? What would the new timetable be then?

Cheers,

Mark

Re: Proposal for the transition timetable for the move to GIT

[Segher Boessenkool]

Hi!

On Mon, Dec 16, 2019 at 10:53:05AM +0100, Mark Wielaard wrote:
> On Fri, 2019-12-06 at 17:44 +0300, Maxim Kuvyrkov wrote:
> > > On Sep 19, 2019, at 6:34 PM, Maxim Kuvyrkov  
> > > wrote:
> > > > On Sep 17, 2019, at 3:02 PM, Richard Earnshaw (lists) 
> > > >  wrote:
> > > > 
> > > > Monday 16th December 2019 - cut off date for picking which git 
> > > > conversion to use
> > > > 
> > > > Tuesday 31st December 2019 - SVN repo becomes read-only at end of stage 
> > > > 3.
> > > > 
> > > > Thursday 2nd January 2020 - (ie read-only + 2 days) new git repo comes 
> > > > on line for live commits.
> > > 
> > > I have regenerated my primary version this week, and it's up at
> > > https://git.linaro.org/people/maxim-kuvyrkov/gcc-pretty.git/ .
> > > So far I have received only minor issue reports about it, and all known 
> > > problems have been fixed.
> > > I could use a bit more scrutiny :-).
> > 
> > I think now is a good time to give status update on the svn->git conversion 
> > I maintain.
> > See https://git.linaro.org/people/maxim-kuvyrkov/gcc-pretty.git/ .
> > 
> > 1. The conversion has all SVN live branches converted as branches under 
> > refs/heads/* .

That is true as far as I can see.  All branches I care about are there,
at least, and I don't see anything missing.

> > 2. The conversion has all SVN live tags converted as annotated tags under 
> > refs/tags/* .

Yup.

> > 3. If desired, it would be trivial to add all deleted / leaf SVN branches 
> > and tags.
> >They would be named as branches/my-deleted-branch@12345,
> >where @12345 is the revision at which the branch was deleted.
> >Branches created and deleted multiple times would have separate entries
> >corresponding to delete revisions.

I don't think this is desirable.

> > 4. Git committer and git author entries are very accurate
> >(imo, better than reposurgeon's, but I'm biased).
> >Developers' names and email addresses are mined from commit logs,
> >changelogs and source code and have historically-accurately attributions
> >to employer's email addresses.

They are very good, yes.  I have verified this *a lot*, months ago.  This
was all ready to go before the Cauldron.

> > 5. Since there is interest in reparenting branches to fix cvs2svn merge 
> > issues,
> >I've added this feature to my scripts as well (turned out to be trivial).
> >I'll keep the original gcc-pretty.git repo intact and will upload the 
> > new one at
> >https://git.linaro.org/people/maxim-kuvyrkov/gcc-reparent.git/
> >-- should be live by Monday.
> 
> Should we go with the gcc-reparent.git repo now?

I don't actually know what the difference is.  As far as I understand it
changes nothing for anything from this century, so either is fine with me.
And it is not very useful to have this old history cleaned up a bit: the
really *big* problem with the old history is that a) people did omnibus
commits a lot, not small self-contained commits changing one thing only;
and b) we really need to have the motivation that goes with those patches,
but that is not available (no mail archives).

> Where exactly should it be installed under https://gcc.gnu.org/git/
> Replacing the existing gcc.git will be confusing, but then how would we
> name the repo that will become the main git gcc repo in 2 weeks?

I think we should rename the old gcc.git mirror.  That pain is temporary.

> Where are the tools/scripts that should be installed on gcc.gnu.org to
> keep it up to date during the next 2 week transition period?

I think Maxim mentioned it before, but it's hard to find in this
humonguous thread :-)


Segher

Re: Proposal for the transition timetable for the move to GIT

[Segher Boessenkool]

On Mon, Dec 16, 2019 at 01:43:48PM +0100, Mark Wielaard wrote:
> On Mon, 2019-12-16 at 11:29 +, Joseph Myers wrote:
> > On Mon, 16 Dec 2019, Mark Wielaard wrote:
> > 
> > > Should we go with the gcc-reparent.git repo now?
> > 
> > I think we should go with the reposurgeon conversion, with all Richard's 
> > improvements to commit messages.  gcc-reparent.git has issues of its own; 
> > at least, checking the list of branches shows some branches are missing.  

You need to provide proof of that.

> > So both conversions can still be considered works in progress.
> 
> I thought we would pick the best available conversion today.

Yes.

> If we keep tweaking the conversions till they are "perfect" we probably
> never reach that point.
> 
> > However, we should also note that stage 3 is intended to last two months, 
> > ending with the move to git 
> >  
> > , and given that it 
> > didn't start at the start of November as anticipated in the originally 
> > proposed timetable, that implies corresponding updates to all the dates.  

I do not agree.

> > By now, enough people are away until the new year that now isn't a good 
> > time for deciding things anyway.
> 
> The idea was to do it while most people were away to have the least
> impact. The timeline https://gcc.gnu.org/wiki/GitConversion does say we
> can slip for logistical reasons the read-only date (2019/12/31) by a
> few days.

Yes.

> Do people really want to keep tweaking the conversions and postpone the
> git switchover?

No.


Segher

C2X Proposal, merge '.' and '->' C operators

[J Decker]

Here's the gist of what I would propose...
https://gist.github.com/d3x0r/f496d0032476ed8b6f980f7ed31280da

In C, there are two operators . and -> used to access members of struct and
union types. These operators are specified such that they are always paired
in usage; for example, if the left hand expression is a pointer to a struct
or union, then the operator -> MUST be used. There is no occasion where .
and -> may be interchanged, given the existing specification.

It should be very evident to the compiler whether the token before '.' or
'->' is a pointer to a struct/union or a struct/union, and just build the
appropriate output.

The source modification for the compiler is very slight, even depending on
flag_c2x(that's not it's name).  It ends up changing a lot of existing
lines, just to change their indentation; but that shouldn't really count
against 'changed lines'.

I'm sure, after 4 score and some years ('78-19) that it must surely have
come up before?  Anyone able to point me to those existing proposals?

D

Re: Code bloat due to silly IRA cost model?

[Georg-Johann Lay]

Am 11.12.19 um 18:55 schrieb Richard Sandiford:

Georg-Johann Lay  writes:

Hi, doesn't actually anybody know know to make memory more expensive
than registers when it comes to allocating registers?

Whatever I am trying for TARGET_MEMORY_MOVE_COST and
TARGET_REGISTER_MOVE_COST, ira-costs.c always makes registers more
expensive than mem and therefore allocates values to stack slots instead
of keeping them in registers.

Test case (for avr) is as simple as it gets:

float func (float);

float call (float f)
{
  return func (f);
}

What am I missing?

Johann


Georg-Johann Lay schrieb:

Hi,

I am trying to track down a code bloat issue and am stuck because I do
not understand IRA's cost model.

The test case is as simple as it gets:

float func (float);

float call (float f)
{
 return func (f);
}

IRA dump shows the following insns:


(insn 14 4 2 2 (set (reg:SF 44)
 (reg:SF 22 r22 [ f ])) "bloat.c":4:1 85 {*movsf}
  (expr_list:REG_DEAD (reg:SF 22 r22 [ f ])
 (nil)))
(insn 2 14 3 2 (set (reg/v:SF 43 [ f ])
 (reg:SF 44)) "bloat.c":4:1 85 {*movsf}
  (expr_list:REG_DEAD (reg:SF 44)
 (nil)))
(note 3 2 6 2 NOTE_INSN_FUNCTION_BEG)
(insn 6 3 7 2 (set (reg:SF 22 r22)
 (reg/v:SF 43 [ f ])) "bloat.c":5:12 85 {*movsf}
  (expr_list:REG_DEAD (reg/v:SF 43 [ f ])
 (nil)))
(call_insn/j 7 6 8 2 (parallel [

#14 sets pseudo 44 from arg register R22.
#2 moves it to pseudo 43
#6 moves it to R22 as it prepares for call_insn #7.

There are 2 allocnos and cost:

Pass 0 for finding pseudo/allocno costs

 a1 (r44,l0) best NO_REGS, allocno NO_REGS
 a0 (r43,l0) best NO_REGS, allocno NO_REGS

   a0(r43,l0) costs: ADDW_REGS:32000 SIMPLE_LD_REGS:32000 LD_REGS:32000
NO_LD_REGS:32000 GENERAL_REGS:32000 MEM:9000
   a1(r44,l0) costs: ADDW_REGS:32000 SIMPLE_LD_REGS:32000 LD_REGS:32000
NO_LD_REGS:32000 GENERAL_REGS:32000 MEM:9000

which is quite odd because MEM is way more expensive here than any REG.

Okay, so let's boost the MEM cost (TARGET_MEMORY_MOVE_COST) by a factor
of 100:

 a1 (r44,l0) best NO_REGS, allocno NO_REGS
 a0 (r43,l0) best NO_REGS, allocno NO_REGS

   a0(r43,l0) costs: ADDW_REGS:320 SIMPLE_LD_REGS:320
LD_REGS:320 NO_LD_REGS:320 GENERAL_REGS:320 MEM:801000
   a1(r44,l0) costs: ADDW_REGS:320 SIMPLE_LD_REGS:320
LD_REGS:320 NO_LD_REGS:320 GENERAL_REGS:320 MEM:801000

What??? The REG costs are 100 times higher, and stille higher that the
MEM costs.  What the heck is going on?

Setting TARGET_REGISTER_MOVE_COST and also TARGET_MEMORY_MOVE_COST to 0
yiels:

   a0(r43,l0) costs: ADDW_REGS:0 SIMPLE_LD_REGS:0 LD_REGS:0 NO_LD_REGS:0
GENERAL_REGS:0 MEM:0
   a1(r44,l0) costs: ADDW_REGS:0 SIMPLE_LD_REGS:0 LD_REGS:0 NO_LD_REGS:0
GENERAL_REGS:0 MEM:0

as expected, i.e. there is no other hidden source of costs considered by
IRA.  And even TARGET_REGISTER_MOVE_COST = 0  and
TARGET_MEMORY_MOVE_COST = original gives:

   a0(r43,l0) costs: ADDW_REGS:32000 SIMPLE_LD_REGS:32000 LD_REGS:32000
NO_LD_REGS:32000 GENERAL_REGS:32000 MEM:9000
   a1(r44,l0) costs: ADDW_REGS:32000 SIMPLE_LD_REGS:32000 LD_REGS:32000
NO_LD_REGS:32000 GENERAL_REGS:32000 MEM:9000

How the heck do I tell ira-costs that registers are way cheaper than MEM?


I think this is coming from:

   /* FIXME: Ideally, the following test is not needed.
 However, it turned out that it can reduce the number
 of spill fails.  AVR and it's poor endowment with
 address registers is extreme stress test for reload.  */

   if (GET_MODE_SIZE (mode) >= 4
   && regno >= REG_X)
 return false;

in avr_hard_regno_mode_ok.  This forbids SFmode in r26+ and means that
moves between pointer registers and general registers have the highest
possible cost (65535) to prevent them for being used for SFmode.  So:

ira_register_move_cost[SFmode][POINTER_REGS][GENERAL_REGS] = 65535;

The costs for union classes are the maximum (worst-case) cost of
for each subclass, so this means that:

ira_register_move_cost[SFmode][GENERAL_REGS][GENERAL_REGS] = 65535;

as well.

Removing the code above fixes it.  If you don't want to do that, an
alternative might be to add a class for r0-r25 (but I've not tested that).


I am still having some headache understanding that...

For example, currently R26 is forbidden for SFmode, but the same applies 
to R25 or any odd registers (modes >= 2 regs have to start in even 
registers).


Then this would imply, even after the condition regno >= 26 was removed, 
the costs would still be astronomically high because HI:21 is refused 
and SI:23 is refused etc, and due to that the cost of that class will be 
0x1 for modes >= 2 regs?


How can the register allocator tell apart whether a register is rejected 
due to its mode or due to the register number?  AFAIK there is no other 
ws than rejecting odd registers in that hook, because register classes 
must not have holes.  Or did that change meanwhile?


Johann



Thanks,
Richard

Re: Proposal for the transition timetable for the move to GIT

[Joseph Myers]

On Mon, 16 Dec 2019, Mark Wielaard wrote:

> > I think we should go with the reposurgeon conversion, with all Richard's 
> > improvements to commit messages.  gcc-reparent.git has issues of its own; 
> > at least, checking the list of branches shows some branches are missing.  
> > So both conversions can still be considered works in progress.
> 
> I thought we would pick the best available conversion today.
> If we keep tweaking the conversions till they are "perfect" we probably
> never reach that point.

There is a difference between tweaking until they are perfect, and 
allowing a little more time to get major improvements for which code 
actually exists (Richard's commit message scripts).

If the go port of reposurgeon were still unfinished, or there were major 
problems with the conversion that weren't understood, considerations might 
be different.  But the go port is fully functional and all the known 
issues affecting tree contents are fixed; to the extent there are other 
issues, they are less significant and also well-understood and should be 
fixed soon.

> > However, we should also note that stage 3 is intended to last two months, 
> > ending with the move to git 
> >  
> > , and given that it 
> > didn't start at the start of November as anticipated in the originally 
> > proposed timetable, that implies corresponding updates to all the dates.  
> > By now, enough people are away until the new year that now isn't a good 
> > time for deciding things anyway.
> 
> The idea was to do it while most people were away to have the least
> impact. The timeline https://gcc.gnu.org/wiki/GitConversion does say we
> can slip for logistical reasons the read-only date (2019/12/31) by a
> few days.

It was also that doing it at the end of stage 3 would mean the least 
disruption to development for stage 3.  That suggests converting over the 
weekend of 18/19 January, given the current stage 3 timings.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Proposal for the transition timetable for the move to GIT

[Eric S. Raymond]

Segher Boessenkool :
> > Do people really want to keep tweaking the conversions and postpone the
> > git switchover?
> 
> No.

It may not be my place to say, but...I think the stakes are pretty
high here.  If I were a GCC developer, I think I'd want the best
possible conversion even if that takes a little longer.

jsm28, rearnsha, and my reposurgeon crew are pretty close to a final
deliverable now. We know what the remaining issues are, they're not
major, and we have a strategy for fixing them. Have a little patience,
please.

Better yet, come over to #reposurgeon on freenode and help out. Anyone
who can run tests on a machine with >128GB RAM would be especially
welcome.
-- 
http://www.catb.org/~esr/";>Eric S. Raymond

Re: Proposal for the transition timetable for the move to GIT

[Joseph Myers]

On Mon, 16 Dec 2019, Segher Boessenkool wrote:

> On Mon, Dec 16, 2019 at 01:43:48PM +0100, Mark Wielaard wrote:
> > On Mon, 2019-12-16 at 11:29 +, Joseph Myers wrote:
> > > On Mon, 16 Dec 2019, Mark Wielaard wrote:
> > > 
> > > > Should we go with the gcc-reparent.git repo now?
> > > 
> > > I think we should go with the reposurgeon conversion, with all Richard's 
> > > improvements to commit messages.  gcc-reparent.git has issues of its own; 
> > > at least, checking the list of branches shows some branches are missing.  
> 
> You need to provide proof of that.

classpath-generics gcj/classpath-095-import-branch libstdcxx_so_7-2-branch 
st/binutils st/mono-based-binutils.  There are also tags in the GCC SVN 
repository in /branches/st/tags, some of which are missing in the 
conversion and the rest of which are in refs/heads/ when refs/tags/ would 
be more appropriate.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Proposal for the transition timetable for the move to GIT

[Segher Boessenkool]

On Mon, Dec 16, 2019 at 08:54:51AM -0500, Eric S. Raymond wrote:
> Segher Boessenkool :
> > > Do people really want to keep tweaking the conversions and postpone the
> > > git switchover?
> > 
> > No.
> 
> It may not be my place to say, but...I think the stakes are pretty
> high here.  If I were a GCC developer, I think I'd want the best
> possible conversion even if that takes a little longer.

Most of us are perfectly happy even with the current git mirror, for
old commits.  We want "real" git to make the workflow for new commits
better.

No more delays, _please_.

If the reposurgeon conversion is not ready now, then it is too late
to be selected.


Segher

Re: Proposal for the transition timetable for the move to GIT

[Joseph Myers]

On Mon, 16 Dec 2019, Segher Boessenkool wrote:

> Most of us are perfectly happy even with the current git mirror, for
> old commits.  We want "real" git to make the workflow for new commits
> better.
> 
> No more delays, _please_.

The timetable is a useful guideline.  It should not be our master when 
there are clear improvements with implementations already available; 
waiting to the actual end of stage 3 makes sense (when waiting another 
year would not make sense).  When we're talking about something to be used 
for the next 20 years we should make sure to get it right.

All conversions clearly need more validation work.  That missing branches 
in Maxim's conversion could be noted only today clearly shows that 
validation of that conversion is also at a very early stage (and 
conversions with an ad hoc script need much more thorough, trickier 
validation because you don't benefit from knowing the tool has worked for 
other conversions).

> If the reposurgeon conversion is not ready now, then it is too late
> to be selected.

I believe it's at least as ready as Maxim's.  The last public version has 
some known issues, most of those have been addressed since that conversion 
run, others are being addressed.  I fully expect it would in fact be in a 
good state to run the final conversion on the original dates, even though 
those are before the end of stage 3.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Code bloat due to silly IRA cost model?

[Richard Sandiford]

Georg-Johann Lay  writes:
> Am 11.12.19 um 18:55 schrieb Richard Sandiford:
>> Georg-Johann Lay  writes:
>>> Hi, doesn't actually anybody know know to make memory more expensive
>>> than registers when it comes to allocating registers?
>>>
>>> Whatever I am trying for TARGET_MEMORY_MOVE_COST and
>>> TARGET_REGISTER_MOVE_COST, ira-costs.c always makes registers more
>>> expensive than mem and therefore allocates values to stack slots instead
>>> of keeping them in registers.
>>>
>>> Test case (for avr) is as simple as it gets:
>>>
>>> float func (float);
>>>
>>> float call (float f)
>>> {
>>>   return func (f);
>>> }
>>>
>>> What am I missing?
>>>
>>> Johann
>>>
>>>
>>> Georg-Johann Lay schrieb:
 Hi,

 I am trying to track down a code bloat issue and am stuck because I do
 not understand IRA's cost model.

 The test case is as simple as it gets:

 float func (float);

 float call (float f)
 {
  return func (f);
 }

 IRA dump shows the following insns:


 (insn 14 4 2 2 (set (reg:SF 44)
  (reg:SF 22 r22 [ f ])) "bloat.c":4:1 85 {*movsf}
   (expr_list:REG_DEAD (reg:SF 22 r22 [ f ])
  (nil)))
 (insn 2 14 3 2 (set (reg/v:SF 43 [ f ])
  (reg:SF 44)) "bloat.c":4:1 85 {*movsf}
   (expr_list:REG_DEAD (reg:SF 44)
  (nil)))
 (note 3 2 6 2 NOTE_INSN_FUNCTION_BEG)
 (insn 6 3 7 2 (set (reg:SF 22 r22)
  (reg/v:SF 43 [ f ])) "bloat.c":5:12 85 {*movsf}
   (expr_list:REG_DEAD (reg/v:SF 43 [ f ])
  (nil)))
 (call_insn/j 7 6 8 2 (parallel [

 #14 sets pseudo 44 from arg register R22.
 #2 moves it to pseudo 43
 #6 moves it to R22 as it prepares for call_insn #7.

 There are 2 allocnos and cost:

 Pass 0 for finding pseudo/allocno costs

  a1 (r44,l0) best NO_REGS, allocno NO_REGS
  a0 (r43,l0) best NO_REGS, allocno NO_REGS

a0(r43,l0) costs: ADDW_REGS:32000 SIMPLE_LD_REGS:32000 LD_REGS:32000
 NO_LD_REGS:32000 GENERAL_REGS:32000 MEM:9000
a1(r44,l0) costs: ADDW_REGS:32000 SIMPLE_LD_REGS:32000 LD_REGS:32000
 NO_LD_REGS:32000 GENERAL_REGS:32000 MEM:9000

 which is quite odd because MEM is way more expensive here than any REG.

 Okay, so let's boost the MEM cost (TARGET_MEMORY_MOVE_COST) by a factor
 of 100:

  a1 (r44,l0) best NO_REGS, allocno NO_REGS
  a0 (r43,l0) best NO_REGS, allocno NO_REGS

a0(r43,l0) costs: ADDW_REGS:320 SIMPLE_LD_REGS:320
 LD_REGS:320 NO_LD_REGS:320 GENERAL_REGS:320 MEM:801000
a1(r44,l0) costs: ADDW_REGS:320 SIMPLE_LD_REGS:320
 LD_REGS:320 NO_LD_REGS:320 GENERAL_REGS:320 MEM:801000

 What??? The REG costs are 100 times higher, and stille higher that the
 MEM costs.  What the heck is going on?

 Setting TARGET_REGISTER_MOVE_COST and also TARGET_MEMORY_MOVE_COST to 0
 yiels:

a0(r43,l0) costs: ADDW_REGS:0 SIMPLE_LD_REGS:0 LD_REGS:0 NO_LD_REGS:0
 GENERAL_REGS:0 MEM:0
a1(r44,l0) costs: ADDW_REGS:0 SIMPLE_LD_REGS:0 LD_REGS:0 NO_LD_REGS:0
 GENERAL_REGS:0 MEM:0

 as expected, i.e. there is no other hidden source of costs considered by
 IRA.  And even TARGET_REGISTER_MOVE_COST = 0  and
 TARGET_MEMORY_MOVE_COST = original gives:

a0(r43,l0) costs: ADDW_REGS:32000 SIMPLE_LD_REGS:32000 LD_REGS:32000
 NO_LD_REGS:32000 GENERAL_REGS:32000 MEM:9000
a1(r44,l0) costs: ADDW_REGS:32000 SIMPLE_LD_REGS:32000 LD_REGS:32000
 NO_LD_REGS:32000 GENERAL_REGS:32000 MEM:9000

 How the heck do I tell ira-costs that registers are way cheaper than MEM?
>> 
>> I think this is coming from:
>> 
>>/* FIXME: Ideally, the following test is not needed.
>>  However, it turned out that it can reduce the number
>>  of spill fails.  AVR and it's poor endowment with
>>  address registers is extreme stress test for reload.  */
>> 
>>if (GET_MODE_SIZE (mode) >= 4
>>&& regno >= REG_X)
>>  return false;
>> 
>> in avr_hard_regno_mode_ok.  This forbids SFmode in r26+ and means that
>> moves between pointer registers and general registers have the highest
>> possible cost (65535) to prevent them for being used for SFmode.  So:
>> 
>> ira_register_move_cost[SFmode][POINTER_REGS][GENERAL_REGS] = 65535;
>> 
>> The costs for union classes are the maximum (worst-case) cost of
>> for each subclass, so this means that:
>> 
>> ira_register_move_cost[SFmode][GENERAL_REGS][GENERAL_REGS] = 65535;
>> 
>> as well.
>> 
>> Removing the code above fixes it.  If you don't want to do that, an
>> alternative might be to add a class for r0-r25 (but I've not tested that).
>
> I am still having some headache understanding that...
>
> For example, currently R26 is forbidden for SFmode, but the same applies 
> to R25 or any odd registers (modes >= 2 regs 

Re: Proposal for the transition timetable for the move to GIT

[Mark Wielaard]

On Mon, 2019-12-16 at 13:56 +, Joseph Myers wrote:
> classpath-generics gcj/classpath-095-import-branch
> libstdcxx_so_7-2-branch st/binutils st/mono-based-binutils.

The classpath "branches" should not be in the final git repo. Those
"branches" are really separate from the actual gcc code tree.

Cheers,

Mark

Re: Proposal for the transition timetable for the move to GIT

[Segher Boessenkool]

On Mon, Dec 16, 2019 at 02:13:06PM +, Joseph Myers wrote:
> On Mon, 16 Dec 2019, Segher Boessenkool wrote:
> 
> > Most of us are perfectly happy even with the current git mirror, for
> > old commits.  We want "real" git to make the workflow for new commits
> > better.
> > 
> > No more delays, _please_.
> 
> The timetable is a useful guideline.  It should not be our master when 
> there are clear improvements with implementations already available; 
> waiting to the actual end of stage 3 makes sense (when waiting another 
> year would not make sense).  When we're talking about something to be used 
> for the next 20 years we should make sure to get it right.

We should not take five years to get it done.

And the current mirror is "right", already, as Jeff said at the Cauldron
(a minute before we unanymously decided to do the conversion soon; this
is over three months ago already).

> All conversions clearly need more validation work.

No, I do not agree with that.  We have had the opportunity to look at
Maxim's conversions for months already, since before the Cauldron, and
it has been perfectly adequate from the start imnsho, and it has been
improved a little since even.

> That missing branches 
> in Maxim's conversion could be noted only today clearly shows that 

... clearly shows that *no one cares* about those branches.

 (and 
> conversions with an ad hoc script need much more thorough, trickier 
> validation because you don't benefit from knowing the tool has worked for 
> other conversions).

Reposurgeon is ad-hoc as well, and the current implementation is a
complete rewrite, and not proven *at all*.  At least Maxim's scripts are
just that: scripts, using some very well-tested very widely used tools
as building blocks.

> > If the reposurgeon conversion is not ready now, then it is too late
> > to be selected.
> 
> I believe it's at least as ready as Maxim's.

I do not agree.  You say the reposurgeon conversion is not ready today.
Maxim's conversion has been ready for many months.


Segher

Re: Proposal for the transition timetable for the move to GIT

[Jeff Law]

On Mon, 2019-12-16 at 08:54 -0500, Eric S. Raymond wrote:
> Segher Boessenkool :
> > > Do people really want to keep tweaking the conversions and postpone the
> > > git switchover?
> > 
> > No.
> 
> It may not be my place to say, but...I think the stakes are pretty
> high here.  If I were a GCC developer, I think I'd want the best
> possible conversion even if that takes a little longer.
Well, I'm not sure that's entirely true.

I do a ton of historical digging, possibly more than anyone else
involved with GCC.  The git-svn mirror has been sufficient for that for
years, even with the warts that folks have pointed out.

Given that, delaying to achieve a perfect conversion is, IMHO, just
silly.  I don't mind delaying a few days here or there because we want
to do verification, or to line up better with our own development
schedules.  What I don't want to do is delay because any particular
tool is still being tweaked to get closer to that "perfect" conversion.

So the question I would ask is the state of each converter today and
how they compare to each other.  That argues we need time to compare
the result, which as I noted above is fine by me.  But we ought to be
comparing the converter's state as of right now.

Jeff

Re: Proposal for the transition timetable for the move to GIT

[Eric S. Raymond]

Joseph Myers :
> When we're talking about something to be used 
> for the next 20 years we should make sure to get it right.

Segher and others should note that I'm not in the habit of sinking most of
a year of my time into problems that I don't think are extremely
important. This conversion *is* that important.

> conversions with an ad hoc script need much more thorough, trickier 
> validation because you don't benefit from knowing the tool has worked for 
> other conversions).

Nor, as far as I am aware, do the scripts have anything resembling
reposurgeon's test suite.

Segher Boessenkool:
> > If the reposurgeon conversion is not ready now, then it is too late
> > to be selected.

Maxim's conversion pipeline isn't ready either -- there are known
bugs with its result. Does that mean it's too late to select Maxim's
conversion? If so, what do you propose be done?

Please stop bellyaching and pitch in. Whether it's by fixing up
Maxim's conversion, helping improve the reposurgeon one,
or writing a conversion method of your own - I don't much care
and it's not my job to tell you what to do, anyway. Any of those
choices might be helpful; sniping from the sidelines is not.
-- 
http://www.catb.org/~esr/";>Eric S. Raymond

Re: Proposal for the transition timetable for the move to GIT

[Joseph Myers]

On Mon, 16 Dec 2019, Mark Wielaard wrote:

> On Mon, 2019-12-16 at 13:56 +, Joseph Myers wrote:
> > classpath-generics gcj/classpath-095-import-branch
> > libstdcxx_so_7-2-branch st/binutils st/mono-based-binutils.
> 
> The classpath "branches" should not be in the final git repo. Those
> "branches" are really separate from the actual gcc code tree.

They're branches of part of GCC (and duly have their contents in a 
libjava/ subdirectory).  I think something present as a non-deleted branch 
in the GCC repository is appropriate for the conversion.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Proposal for the transition timetable for the move to GIT

[Joseph Myers]

On Mon, 16 Dec 2019, Segher Boessenkool wrote:

> And the current mirror is "right", already, as Jeff said at the Cauldron
> (a minute before we unanymously decided to do the conversion soon; this
> is over three months ago already).

All the discussion at the Cauldron tells us is many people like the idea 
of moving to git and it not taking forever.  Inviting a crowd to agree 
with a proposition is not a useful way to judge the technical merits of 
any particular detailed conversion approach.

> > That missing branches 
> > in Maxim's conversion could be noted only today clearly shows that 
> 
> ... clearly shows that *no one cares* about those branches.

Since Maxim said that all branches were present, that indicates a lack of 
validation, and a lack of validation by other people since then.  Checking 
the set of branches and tags present is one of the most basic checks on a 
conversion to identify problems.

> > I believe it's at least as ready as Maxim's.
> 
> I do not agree.  You say the reposurgeon conversion is not ready today.
> Maxim's conversion has been ready for many months.

I believe it's ready in the form of source code (gcc-conversion repository 
and newsvn3 branch in the reposurgeon repository).  I'm running a test 
conversion to check this and produce the binary form (converted git 
repository); conversions just take a while to run.  With correctness 
issues having been addressed, we're working on performance issues, and I'm 
running a second test conversion on a second machine with both a patch 
I've just written that passes reposurgeon's tests and I hope will save 
about 8 hours on the conversion time, and further performance improvements 
that went in overnight that should save some more hours via saving memory 
usage.  (A significant proportion of the time for a conversion is spent by 
git-fast-import reading the fast-import stream, which places a lower bound 
of a few hours on the time taken for a conversion even if everything 
outside of git is infinitely fast.)

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Proposal for the transition timetable for the move to GIT

[Eric S. Raymond]

Jeff Law :
> > It may not be my place to say, but...I think the stakes are pretty
> > high here.  If I were a GCC developer, I think I'd want the best
> > possible conversion even if that takes a little longer.
> Well, I'm not sure that's entirely true.

OK, that's a policy choice the GCC project is going to have to make.
I'm just the mechanic here.

Joseph Myers has made his choice.  He has said repeatedly that he
wants to follow through with the reposurgeon conversion, and he's
putting his effort behind that by writing tests and even contributing
code to reposurgeon.

We'll get this done faster if nobody is joggling his elbow. Or mine.
-- 
http://www.catb.org/~esr/";>Eric S. Raymond

Re: Proposal for the transition timetable for the move to GIT

[Jeff Law]

On Mon, 2019-12-16 at 13:53 +, Joseph Myers wrote:
> 
> > > However, we should also note that stage 3 is intended to last two months, 
> > > ending with the move to git 
> > >  
> > > ;, and given that it 
> > > didn't start at the start of November as anticipated in the originally 
> > > proposed timetable, that implies corresponding updates to all the dates.  
> > > By now, enough people are away until the new year that now isn't a good 
> > > time for deciding things anyway.
> > 
> > The idea was to do it while most people were away to have the least
> > impact. The timeline https://gcc.gnu.org/wiki/GitConversion does say we
> > can slip for logistical reasons the read-only date (2019/12/31) by a
> > few days.
> 
> It was also that doing it at the end of stage 3 would mean the least 
> disruption to development for stage 3.  That suggests converting over the 
> weekend of 18/19 January, given the current stage 3 timings.
My recollection was the timing was meant to land roughly at the stage3-
>stage4 transition.  So the question is whether or not we're on target
with that.  Based on the regression counts, probably not at this point,
but I'll let the release managers chime in on that point.

Jeff

Re: Proposal for the transition timetable for the move to GIT

[Jeff Law]

On Mon, 2019-12-16 at 11:37 -0500, Eric S. Raymond wrote:
> Jeff Law :
> > > It may not be my place to say, but...I think the stakes are pretty
> > > high here.  If I were a GCC developer, I think I'd want the best
> > > possible conversion even if that takes a little longer.
> > Well, I'm not sure that's entirely true.
> 
> OK, that's a policy choice the GCC project is going to have to make.
> I'm just the mechanic here.
Yup.  And I wouldn't be surprised if there is dissent regardless of
what final decision is made.
> 
> Joseph Myers has made his choice.  He has said repeatedly that he
> wants to follow through with the reposurgeon conversion, and he's
> putting his effort behind that by writing tests and even contributing
> code to reposurgeon.
> 
> We'll get this done faster if nobody is joggling his elbow. Or mine.
And just to be clear, my preference is for reposurgeon, if it's ready. 
But if it isn't, then I'm absolutely comfortable dropping back to
Maxim's conversion or even the existing mirror.



Jeff

Re: Proposal for the transition timetable for the move to GIT

[Segher Boessenkool]

On Mon, Dec 16, 2019 at 11:27:56AM -0500, Eric S. Raymond wrote:
> Joseph Myers :
> > When we're talking about something to be used 
> > for the next 20 years we should make sure to get it right.
> 
> Segher and others should note that I'm not in the habit of sinking most of
> a year of my time into problems that I don't think are extremely
> important. This conversion *is* that important.

To you, whatever reposurgeon does that nothing else can, is important.
To many other people, not.  Most people are totally pragmatic and want
to use a git-based workflow with GCC, and then soon upgrade some other
things in our workflow, to improve our day-to-day experience, and to
allow us to do things we couldn't do before.

Most people do not care about fixing the imperfections in the CVS->SVN
conversion.  We have been using the SVN->Git mirror for over ten years
now, and it is perfectly workable.  Now we want to finally finally
_FINALLY_ have an actual git repo that we can commit patches to
directly.  Which we unanimously decided to do over three months ago.

> Nor, as far as I am aware, do the scripts have anything resembling
> reposurgeon's test suite.

So?  Such a test suite does not magically prevent bugs (whatever type
it is: regressions, unit tests, whatever methodology).

The only thing that matters is acceptance testing (which includes such
trivial things ass "are all the files on trunk what they should be").

> Segher Boessenkool:
> > > If the reposurgeon conversion is not ready now, then it is too late
> > > to be selected.
> 
> Maxim's conversion pipeline isn't ready either -- there are known
> bugs with its result.

Are there?  The last I heard is that branches that do not share any
history with GCC are not in there.  That's a feature, not a bug, imnsho.

If you know of any other bugs, detail them, don't make unfounded
statements please.

> Does that mean it's too late to select Maxim's
> conversion? If so, what do you propose be done?

Maxim's conversion was perfectly acceptable many months ago already.

> Please stop bellyaching and pitch in. Whether it's by fixing up
> Maxim's conversion, helping improve the reposurgeon one,
> or writing a conversion method of your own - I don't much care
> and it's not my job to tell you what to do, anyway. Any of those
> choices might be helpful; sniping from the sidelines is not.

Lol.  Yeah, I won't answer that at all, I guess.


Segher

Re: Proposal for the transition timetable for the move to GIT

[Jeff Law]

On Mon, 2019-12-16 at 11:29 +, Joseph Myers wrote:
> On Mon, 16 Dec 2019, Mark Wielaard wrote:
> 
> > Should we go with the gcc-reparent.git repo now?
> 
> I think we should go with the reposurgeon conversion, with all Richard's 
> improvements to commit messages.  gcc-reparent.git has issues of its own; 
> at least, checking the list of branches shows some branches are missing.  
> So both conversions can still be considered works in progress.
So it seems like your position is that the reposurgeon conversion is as
good as or better than conversion from Maxim's scripts.

If that's a fair assessment of your position, then my vote would be
that we select reposurgeon based on your assessment, which I absolutely
trust. 

jeff

Re: Proposal for the transition timetable for the move to GIT

[Joseph Myers]

On Mon, 16 Dec 2019, Jeff Law wrote:

> On Mon, 2019-12-16 at 11:29 +, Joseph Myers wrote:
> > On Mon, 16 Dec 2019, Mark Wielaard wrote:
> > 
> > > Should we go with the gcc-reparent.git repo now?
> > 
> > I think we should go with the reposurgeon conversion, with all Richard's 
> > improvements to commit messages.  gcc-reparent.git has issues of its own; 
> > at least, checking the list of branches shows some branches are missing.  
> > So both conversions can still be considered works in progress.
> So it seems like your position is that the reposurgeon conversion is as
> good as or better than conversion from Maxim's scripts.

Yes.  It should be possible to confirm branch tip conversions and other 
properties of the repository (e.g. that all branch tips are properly 
descended from the first commit on trunk except for the specific branches 
that shouldn't be) once my current conversions have finished running.

I think there may well be things to *learn* from Maxim's conversion to 
improve the reposurgeon one further (if they don't take that long to 
implement).  In particular, we should look carefully at the commit 
attributions in both conversions and Maxim's may well give ideas for 
improving the reposurgeon changelogs command (Richard came up with three 
ideas recently, which I've just filed in the reposurgeon issue tracker).  
But I also think:

* reposurgeon is a safer approach than ad hoc scripts, provided we get 
clean verification of basic properties such as branch tip contents.

* Richard's improvements to commit messages are a great improvement to the 
resulting repository (and it's OK if a small percentage end up misleading 
because someone used the wrong PR number, sometimes people use the wrong 
commit message or commit changes they didn't mean to and so having some 
misleading messages is unavoidable).

* As we're part of the free software community as a whole rather than 
something in isolation, choosing to make a general-purpose tool work for 
our conversion is somewhat preferable to choosing an ad hoc approach 
because it contributes something of value for other repository conversions 
by other projects in future.

-- 
Joseph S. Myers
jos...@codesourcery.com

Git transition

[Richard Earnshaw]

Unfortunately a family emergency has cropped up and I can’t respond directly to 
the thread.  I’m also unlikely to be able to follow up on this discussion in 
the next couple of days.

I proposed the timetable back after the Cauldron to make it clear that we were 
serious about moving this release cycle.  However, the timetable was always a 
bit imprecise and we’ve already moved the final switch over date by a couple of 
weeks, so I don’t think slipping the decision date by a few days will impact 
that cut over.

So my request is please don’t rush to a decision that we might come to regret.  
For a halp’th of tar, and all that.

R.

Re: Proposal for the transition timetable for the move to GIT

[Jeff Law]

On Mon, 2019-12-16 at 09:36 -0600, Segher Boessenkool wrote:
> On Mon, Dec 16, 2019 at 02:13:06PM +, Joseph Myers wrote:
> > On Mon, 16 Dec 2019, Segher Boessenkool wrote:
> > 
> > > Most of us are perfectly happy even with the current git mirror, for
> > > old commits.  We want "real" git to make the workflow for new commits
> > > better.
> > > 
> > > No more delays, _please_.
> > 
> > The timetable is a useful guideline.  It should not be our master when 
> > there are clear improvements with implementations already available; 
> > waiting to the actual end of stage 3 makes sense (when waiting another 
> > year would not make sense).  When we're talking about something to be used 
> > for the next 20 years we should make sure to get it right.
> 
> We should not take five years to get it done.
> 
> And the current mirror is "right", already, as Jeff said at the Cauldron
> (a minute before we unanymously decided to do the conversion soon; this
> is over three months ago already).
Yup.   I want to convert sooner, not later.  I don't mind slipping a
little for validation work or because it doesn't line up with our
schedules.  I don't want to slip for major changes in the tooling.

My preference has always been, in order, reposurgeon, Maxim's scripts
and the existing mirror.  However, I'm confident that all of them will
be sufficient for our needs.


> > All conversions clearly need more validation work.
> 
> No, I do not agree with that.  We have had the opportunity to look at
> Maxim's conversions for months already, since before the Cauldron, and
> it has been perfectly adequate from the start imnsho, and it has been
> improved a little since even.
Yet Joseph just indicated today Maxim's conversion is missing some
branches.  While I don't consider any of the missed branches important,
others might.   More importantly, it raises the issue of what other
branches might be missing and what validation work has been done on
that conversion.


> 
> > That missing branches 
> > in Maxim's conversion could be noted only today clearly shows that 
> 
> ... clearly shows that *no one cares* about those branches.
> 
>  (and 
> > conversions with an ad hoc script need much more thorough, trickier 
> > validation because you don't benefit from knowing the tool has worked for 
> > other conversions).
> 
> Reposurgeon is ad-hoc as well, and the current implementation is a
> complete rewrite, and not proven *at all*.  At least Maxim's scripts are
> just that: scripts, using some very well-tested very widely used tools
> as building blocks.
I wouldn't really classify it that way.  reposurgeon has significant
history.  Are we using a rewrite, yes, but there's extensive experience
behind it as well as testsuites to validate the work.


> 
> > > If the reposurgeon conversion is not ready now, then it is too late
> > > to be selected.
> > 
> > I believe it's at least as ready as Maxim's.
> 
> I do not agree.  You say the reposurgeon conversion is not ready today.
> Maxim's conversion has been ready for many months.
Actually Joseph's position is that reposurgeon is already beyond
Maxim's conversion in terms of quality.

My interpretation of the messages flying by is they're ironing out some
details on the edges, but I don't think those are major intrusive
changes.

jeff

Re: Proposal for the transition timetable for the move to GIT

[Richard Biener]

On December 16, 2019 5:39:06 PM GMT+01:00, Jeff Law  wrote:
>On Mon, 2019-12-16 at 13:53 +, Joseph Myers wrote:
>> 
>> > > However, we should also note that stage 3 is intended to last two
>months, 
>> > > ending with the move to git 
>> > >  
>> > > ;, and given
>that it 
>> > > didn't start at the start of November as anticipated in the
>originally 
>> > > proposed timetable, that implies corresponding updates to all the
>dates.  
>> > > By now, enough people are away until the new year that now isn't
>a good 
>> > > time for deciding things anyway.
>> > 
>> > The idea was to do it while most people were away to have the least
>> > impact. The timeline https://gcc.gnu.org/wiki/GitConversion does
>say we
>> > can slip for logistical reasons the read-only date (2019/12/31) by
>a
>> > few days.
>> 
>> It was also that doing it at the end of stage 3 would mean the least 
>> disruption to development for stage 3.  That suggests converting over
>the 
>> weekend of 18/19 January, given the current stage 3 timings.
>My recollection was the timing was meant to land roughly at the stage3-
>>stage4 transition.  So the question is whether or not we're on target
>with that.  Based on the regression counts, probably not at this point,
>but I'll let the release managers chime in on that point.
>

Fortunately that's a pure timing thing and I'd expect stage3 to end mid 
January, possibly scrapping off a week to help the git conversion. 

Richard. 

>Jeff

Re: Proposal for the transition timetable for the move to GIT

[Eric S. Raymond]

Joseph Myers :
> * As we're part of the free software community as a whole rather than 
> something in isolation, choosing to make a general-purpose tool work for 
> our conversion is somewhat preferable to choosing an ad hoc approach 
> because it contributes something of value for other repository conversions 
> by other projects in future.

That's not just theory or sentiment. Reposurgeon is the best
any-VCS-to-any-VCS converter there is because every time I do a
conversion, I learn things, and that knowledge gets incorporated in
the code and the documentation around it.

Yes, in theory someone else could build a tool as good that
incorporates as much domain knowledge. So far, nobody has tried.  It's
unlikely anyone will, at this point, when they can join my dev team
and get the results they want with much less effort by improving
reposurgeon or one of its auxiliary tools.

Every time that happens, everybody - into the indefinite future - wins.
-- 
http://www.catb.org/~esr/";>Eric S. Raymond

Re: Proposal for the transition timetable for the move to GIT

[Segher Boessenkool]

Hi,

On Mon, Dec 16, 2019 at 05:07:48PM +, Joseph Myers wrote:
> Yes.  It should be possible to confirm branch tip conversions and other 
> properties of the repository (e.g. that all branch tips are properly 
> descended from the first commit on trunk except for the specific branches 
> that shouldn't be) once my current conversions have finished running.

Please do that for Maxim's conversion as well then?

(If the way you do the verification requires reposurgeon, the
verification methodology itself is fatally flawed).

> I think there may well be things to *learn* from Maxim's conversion to 
> improve the reposurgeon one further (if they don't take that long to 
> implement).

Or the other way around.

> In particular, we should look carefully at the commit 
> attributions in both conversions and Maxim's may well give ideas for 
> improving the reposurgeon changelogs command (Richard came up with three 
> ideas recently, which I've just filed in the reposurgeon issue tracker).  
> But I also think:
> 
> * reposurgeon is a safer approach than ad hoc scripts, provided we get 
> clean verification of basic properties such as branch tip contents.

I totally do not agree.  Black boxes are not safe.  *New* black boxes
are even worse.

I trust scripts that have low internal complexity much better.

There is absolutely no reason to trust a system that supposedly was
already very mature, but that required lots of complex modifications,
and even a complete rewrite in a different language, that even has its
own bug tracker, to work without problems (although we all have *seen*
some of its many problems over the last years), and at the same time
bad-mouthing simple scripts that simply work, and have simple problems.

> * Richard's improvements to commit messages are a great improvement to the 
> resulting repository (and it's OK if a small percentage end up misleading 
> because someone used the wrong PR number, sometimes people use the wrong 
> commit message or commit changes they didn't mean to and so having some 
> misleading messages is unavoidable).

As long as the original commit message is kept, verbatim, and you only
add a new summary line, all is fine.  If not -> nope, not okay.

> * As we're part of the free software community as a whole rather than 
> something in isolation, choosing to make a general-purpose tool work for 
> our conversion is somewhat preferable to choosing an ad hoc approach 
> because it contributes something of value for other repository conversions 
> by other projects in future.

This, I don't agree with at all either: having some lean-and-mean
scripts that worked for the GCC conversion will be at least as helpful
for another conversion as a "general" tool that first requires you to
build a custom machine before you can use it properly, would be.


Anyway: yes, please verify all conversion candidates for your criteria.
Thanks!


Segher

Re: Proposal for the transition timetable for the move to GIT

[Jeff Law]

On Mon, 2019-12-16 at 15:59 -0600, Segher Boessenkool wrote:
> In particular, we should look carefully at the commit 
> > attributions in both conversions and Maxim's may well give ideas for 
> > improving the reposurgeon changelogs command (Richard came up with three 
> > ideas recently, which I've just filed in the reposurgeon issue tracker).  
> > But I also think:
> > 
> > * reposurgeon is a safer approach than ad hoc scripts, provided we get 
> > clean verification of basic properties such as branch tip contents.
> 
> I totally do not agree.  Black boxes are not safe.  *New* black boxes
> are even worse.
> 
> I trust scripts that have low internal complexity much better.
Perhaps.  But there's also limits to what scripts can do.

> 
> There is absolutely no reason to trust a system that supposedly was
> already very mature, but that required lots of complex modifications,
> and even a complete rewrite in a different language, that even has its
> own bug tracker, to work without problems (although we all have *seen*
> some of its many problems over the last years), and at the same time
> bad-mouthing simple scripts that simply work, and have simple problems.
I'd disagree.  THe experience and testsuites from that system are a
significant benefit. 

> 
> > * Richard's improvements to commit messages are a great improvement to the 
> > resulting repository (and it's OK if a small percentage end up misleading 
> > because someone used the wrong PR number, sometimes people use the wrong 
> > commit message or commit changes they didn't mean to and so having some 
> > misleading messages is unavoidable).
> 
> As long as the original commit message is kept, verbatim, and you only
> add a new summary line, all is fine.  If not -> nope, not okay.
Sorry, have to disagree here.  I think what Richard has done is a
significant step forward. 


> 
> > * As we're part of the free software community as a whole rather than 
> > something in isolation, choosing to make a general-purpose tool work for 
> > our conversion is somewhat preferable to choosing an ad hoc approach 
> > because it contributes something of value for other repository conversions 
> > by other projects in future.
> 
> This, I don't agree with at all either: having some lean-and-mean
> scripts that worked for the GCC conversion will be at least as helpful
> for another conversion as a "general" tool that first requires you to
> build a custom machine before you can use it properly, would be.
> 
> 
> Anyway: yes, please verify all conversion candidates for your criteria.
> Thanks!
And if they're the same, then I'm still going to prefer reposurgeon.

So unless there's something  Maxim's scripts are getting right that
aren't by reposurgeon, then reposurgeon is the right choice.

jeff

Re: Proposal for the transition timetable for the move to GIT

[Segher Boessenkool]

Hi!

On Mon, Dec 16, 2019 at 03:13:49PM -0700, Jeff Law wrote:
> On Mon, 2019-12-16 at 15:59 -0600, Segher Boessenkool wrote:
> > In particular, we should look carefully at the commit 
> > > attributions in both conversions and Maxim's may well give ideas for 
> > > improving the reposurgeon changelogs command (Richard came up with three 
> > > ideas recently, which I've just filed in the reposurgeon issue tracker).  
> > > But I also think:
> > > 
> > > * reposurgeon is a safer approach than ad hoc scripts, provided we get 
> > > clean verification of basic properties such as branch tip contents.
> > 
> > I totally do not agree.  Black boxes are not safe.  *New* black boxes
> > are even worse.
> > 
> > I trust scripts that have low internal complexity much better.
> Perhaps.  But there's also limits to what scripts can do.

Absolutely.  Just this trust in complicated things is very misplaced, in
my opinion.

> > There is absolutely no reason to trust a system that supposedly was
> > already very mature, but that required lots of complex modifications,
> > and even a complete rewrite in a different language, that even has its
> > own bug tracker, to work without problems (although we all have *seen*
> > some of its many problems over the last years), and at the same time
> > bad-mouthing simple scripts that simply work, and have simple problems.
> I'd disagree.  THe experience and testsuites from that system are a
> significant benefit. 

That isn't what I said.  I said that freshly constructed complex software
will have more and deeper errors than stupid simple scripts do (or I
implied that at least, maybe it wasn't clear).  And I only say this
because the opposite was claimed, which is laughable imnsho.

> > > * Richard's improvements to commit messages are a great improvement to 
> > > the 
> > > resulting repository (and it's OK if a small percentage end up misleading 
> > > because someone used the wrong PR number, sometimes people use the wrong 
> > > commit message or commit changes they didn't mean to and so having some 
> > > misleading messages is unavoidable).
> > 
> > As long as the original commit message is kept, verbatim, and you only
> > add a new summary line, all is fine.  If not -> nope, not okay.
> Sorry, have to disagree here.  I think what Richard has done is a
> significant step forward. 

We talked about it for days, and as far as I understand it Richard agreed.

But, there is no way I can verify this yet, or is there?  Is there a repo
we can look at?  Something close to final.

> > > * As we're part of the free software community as a whole rather than 
> > > something in isolation, choosing to make a general-purpose tool work for 
> > > our conversion is somewhat preferable to choosing an ad hoc approach 
> > > because it contributes something of value for other repository 
> > > conversions 
> > > by other projects in future.
> > 
> > This, I don't agree with at all either: having some lean-and-mean
> > scripts that worked for the GCC conversion will be at least as helpful
> > for another conversion as a "general" tool that first requires you to
> > build a custom machine before you can use it properly, would be.
> > 
> > 
> > Anyway: yes, please verify all conversion candidates for your criteria.
> > Thanks!
> And if they're the same, then I'm still going to prefer reposurgeon.

But they aren't the same, so you will have to make an actual choice,
based on actual data ;-)

> So unless there's something  Maxim's scripts are getting right that
> aren't by reposurgeon, then reposurgeon is the right choice.

... in your opinion.

Anyway, I'd love too hear Richard's input too, but we will have to wait
for that a few days.

The quality of the conversion should be judged by the output of it, not
by anything else.  You do not want to see how sausage is made, as they
say.  The GCC community has nothing to gain from a generic conversion
tool: what we want and need is *one* conversion, and it should be a
good one.

So let us compare *that*!


Segher

Re: Proposal for the transition timetable for the move to GIT

[Joseph Myers]

On Mon, 16 Dec 2019, Segher Boessenkool wrote:

> Hi,
> 
> On Mon, Dec 16, 2019 at 05:07:48PM +, Joseph Myers wrote:
> > Yes.  It should be possible to confirm branch tip conversions and other 
> > properties of the repository (e.g. that all branch tips are properly 
> > descended from the first commit on trunk except for the specific branches 
> > that shouldn't be) once my current conversions have finished running.
> 
> Please do that for Maxim's conversion as well then?
> 
> (If the way you do the verification requires reposurgeon, the
> verification methodology itself is fatally flawed).

It does not require reposurgeon.  The inputs for verification of branch 
tips are (a) a list of correspondences between SVN paths and git refs and 
(b) the SVN revision number at which those refs should correspond to those 
SVN paths.  A list can readily be generated for any git repository not 
using too complicated a mapping from SVN branch names (with a more 
complicated mapping, it's natural for the process modifying the names also 
to generate the list for use in verification).

> > * Richard's improvements to commit messages are a great improvement to the 
> > resulting repository (and it's OK if a small percentage end up misleading 
> > because someone used the wrong PR number, sometimes people use the wrong 
> > commit message or commit changes they didn't mean to and so having some 
> > misleading messages is unavoidable).
> 
> As long as the original commit message is kept, verbatim, and you only
> add a new summary line, all is fine.  If not -> nope, not okay.

That is how it works.  A new summary line is added, with the original 
message kept verbatim after that.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Proposal for the transition timetable for the move to GIT

[Eric S. Raymond]

Segher Boessenkool :
> There is absolutely no reason to trust a system that supposedly was
> already very mature, but that required lots of complex modifications,
> and even a complete rewrite in a different language, that even has its
> own bug tracker, to work without problems (although we all have *seen*
> some of its many problems over the last years), and at the same time
> bad-mouthing simple scripts that simply work, and have simple problems.

Some factual corrections:

I didn't port to Go to fix bugs, I ported for better performance.
Python is a wonderful language for prototyping a tool like this, but
it's too slow and memory-hungry for use at the GCC conversion's
scale.  Also doesn't parallelize worth a damn.

I very carefully *didn't* bad-mouth Maxim's scripts - in facrt I have
said on-list that I think his approach is on the whole pretty
intelligent. To anyone who didn't have some of the experiences I have
had, even using git-svn to analyze basic blocks would appear
reasonable and I don't actually fault Maxim for it.

I *did* bad-mouth git-svn - and I will continue to do so until it no
longer troubles the world with botched conversions.  Relying on it is,
in my subject-matter-expert opinion, unacceptably risky. While I don't
blame Maxim for not being aware of this, it remains a serious
vulnerability in his pipeline.

I don't know how it is on your planet, but here on Earth having a
bug tracker - and keeping it reasonably clean - is generally 
considered a sign of responsible maintainership.


In conclusion, I'm happy that you're so concerned about bugs in
reposurgeon. I am too. You're welcome to file issues and help us
improve our already-extensive test suite by shipping us dumps that
produce errors.
-- 
http://www.catb.org/~esr/";>Eric S. Raymond

Question about struct identifiers after modifications.

[Erick Ochoa]

Hello,

I am working on a struct reorganization optimization pass.
I am able to identify which structs I want to reorganize
and I am also able to create new structs with these modifications.
The way the new structs are generated is the following code
(I am keeping it high-level for conciseness but feel free to ask
for more details).

static tree
get_sorted_record(tree record)
{
  gcc_assert(TREE_CODE(record) == RECORD_TYPE);
  log(2, "printing original");
  print_record(record);
  tree copy = copy_record(record);
  sort_fields_in_place(copy);
  log(2, "printing sorted copy");
  print_record(copy);
  return copy;
}

Output:

[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_record(tree):183)printing record aStruct
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)e,boolean_type,8
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)a,integer_type,32
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)b,integer_type,8
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)c,real_type,32
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)d,real_type,64
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)f,boolean_type,8
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)h,integer_type,64
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:tree_node* 
get_sorted_record(tree):218)printing sorted copy
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_record(tree):183)printing record aStruct
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)e,boolean_type,8
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)b,integer_type,8
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)f,boolean_type,8
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)a,integer_type,32
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)c,real_type,32
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)d,real_type,64
[log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
print_field(tree):199)h,integer_type,64

Basically, we copy the tree of structs of interest.
We modify the DECL_CHAIN so that the fields are in the order that
we want. And finally, we would like to replace trees of type `record`
with `copy`.

At the moment, IDENTIFIER_POINTER for copy and record are the same.
However, I want to keep my design as general as possible.
If I understand correctly, with the current design it is possible
to change all trees of type record with those of type copy.
However, if one wished to change only a subset of these trees,
wouldn't there be a need for a different IDENTIFIER_NODE in clone?
Otherwise, there would be confusion as to how clone and record
differ. (Their IDENTIFIER_POINTERS are the same.)

So, is the correct way to specialize a type is to modify the
identifier myself or is there an API that allows me to do so?

Thanks!

Re: Proposal for the transition timetable for the move to GIT

[Joseph Myers]

On Mon, 16 Dec 2019, Segher Boessenkool wrote:

> We talked about it for days, and as far as I understand it Richard agreed.
> 
> But, there is no way I can verify this yet, or is there?  Is there a repo
> we can look at?  Something close to final.

The conversion run I started this afternoon is now running 
git-fast-import.  That part takes about four hours because that's how long 
git-fast-import takes to process a 240 GB file with all the GCC history 
(and then git gc --aggressive, to get the repository down to a reasonable 
size, takes an hour or two).

So tomorrow I expect to have a repository ready to make available.  (In 
fact, two variants - one with branch and tag names essentially as in SVN, 
one with rearrangements done as suggested by Richard.)  And that will be 
along with the results of validation, which I hope will be clean as we've 
fixed all the issues shown up in previous rounds of validation.

This includes the part of Richard's commit message improvements deduced 
purely from the existing commit messages.  It doesn't include the part 
based on Bugzilla data (because the attempt to download the data from 
Bugzilla fell over).  I expect the next conversion run, started after that 
one finishes, to include both parts of Richard's commit message 
improvements, as well as an improvement to commit attribution extraction 
from ChangeLog files (to include attributions from ChangeLog. 
files, not just plain ChangeLog).

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Proposal for the transition timetable for the move to GIT

[Eric S. Raymond]

Jeff Law :
> So unless there's something  Maxim's scripts are getting right that
> aren't by reposurgeon, then reposurgeon is the right choice.

It is still possible that the scripts could get things right that
reposurgeon doesn't. But the reverse question is also valid. Can
Maxim's scripts get everything right that reposurgeon does?

If anyone wants to audit for that, my test suite is open source.  May
the best program win!
-- 
http://www.catb.org/~esr/";>Eric S. Raymond

Re: Question about struct identifiers after modifications.

[Andrew Pinski]

On Mon, Dec 16, 2019 at 3:24 PM Erick Ochoa
 wrote:
>
> Hello,
>
> I am working on a struct reorganization optimization pass.
> I am able to identify which structs I want to reorganize
> and I am also able to create new structs with these modifications.
> The way the new structs are generated is the following code
> (I am keeping it high-level for conciseness but feel free to ask
> for more details).
>
> static tree
> get_sorted_record(tree record)
> {
>   gcc_assert(TREE_CODE(record) == RECORD_TYPE);
>   log(2, "printing original");
>   print_record(record);
>   tree copy = copy_record(record);
>   sort_fields_in_place(copy);
>   log(2, "printing sorted copy");
>   print_record(copy);
>   return copy;
> }
>
> Output:
>
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_record(tree):183)printing record aStruct
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)e,boolean_type,8
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)a,integer_type,32
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)b,integer_type,8
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)c,real_type,32
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)d,real_type,64
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)f,boolean_type,8
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)h,integer_type,64
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:tree_node* 
> get_sorted_record(tree):218)printing sorted copy
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_record(tree):183)printing record aStruct
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)e,boolean_type,8
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)b,integer_type,8
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)f,boolean_type,8
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)a,integer_type,32
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)c,real_type,32
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)d,real_type,64
> [log](/home/eochoa/code/gcc/gcc/ipa-hello-world.c:void 
> print_field(tree):199)h,integer_type,64
>
> Basically, we copy the tree of structs of interest.
> We modify the DECL_CHAIN so that the fields are in the order that
> we want. And finally, we would like to replace trees of type `record`
> with `copy`.
>
> At the moment, IDENTIFIER_POINTER for copy and record are the same.
> However, I want to keep my design as general as possible.
> If I understand correctly, with the current design it is possible
> to change all trees of type record with those of type copy.
> However, if one wished to change only a subset of these trees,
> wouldn't there be a need for a different IDENTIFIER_NODE in clone?
> Otherwise, there would be confusion as to how clone and record
> differ. (Their IDENTIFIER_POINTERS are the same.)
>
> So, is the correct way to specialize a type is to modify the
> identifier myself or is there an API that allows me to do so?

This is what I did:
+  if (TYPE_NAME (type) != NULL)
+{
+  if (TREE_CODE (TYPE_NAME (type)) == IDENTIFIER_NODE)
+tname = IDENTIFIER_POINTER (TYPE_NAME (type));
+  else if (DECL_NAME (TYPE_NAME (type)) != NULL)
+tname = IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME (type)));
+}
...
+  char id[10];
+  sprintf(id, "%d", i);
+  if (tname)
+ {
+  name = concat (tname, ".reorg.", id, NULL);
+  TYPE_NAME (newtype[i]) = get_identifier (name);
+  free (name);
+}

Where i is the nth new type for the original type.

For the fields that have a new type, I did this:
+  if (nt[1] != NULL && DECL_NAME (fielddecl))
+ {
+   const char *tname = IDENTIFIER_POINTER (DECL_NAME (fielddecl));
+   char id[10];
+   char *name;
+
+   sprintf(id, "%d", i);
+   name = concat (tname, ".reorg.", id, NULL);
+   DECL_NAME (field) = get_identifier (name);
+   free (name);
+ }
+  else
+ DECL_NAME (field) = DECL_NAME (fielddecl);

Where nt was the new types for that field.

Thanks,
Andrew Pinski

>
> Thanks!

Re: Proposal for the transition timetable for the move to GIT

[Eric S. Raymond]

Joseph Myers :
> I expect the next conversion run, started after that 
> one finishes, to include both parts of Richard's commit message 
> improvements, as well as an improvement to commit attribution extraction 
> from ChangeLog files (to include attributions from ChangeLog. 
> files, not just plain ChangeLog).

There is also a known but minor bug in ChangeLog mining at branch roots.
I'm working on that and expect to have a fix shortly.
-- 
http://www.catb.org/~esr/";>Eric S. Raymond

Re: C2X Proposal, merge '.' and '->' C operators

[J Decker]

This is a view of the patch/diff... This is really just +6 lines... ` if(
!flag_iso2xc) `{` `}` `attribute fallthrough`   `if(flag_iso2cx)` `return
ptr`

https://github.com/gcc-mirror/gcc/pull/41/commits/915bcffdea0aa4fead66c41830b66aa3db212307


While the compiler does compile itself, and a simple test case,
 successfully

```
#include 

struct s {
int a, b;
};

void f( void ){
struct s S;
struct s *P = &S;
P.a = 5; // 'wrong' operator
P.b = 13;  // 'wrong' operator
printf( "Output: %d %d\n", S->a, S->b );  // 'wrong' operators...
}

int main( void ) {
f();
return 0;
}
```

I haven't built the testsuite...


On Mon, Dec 16, 2019 at 5:51 AM J Decker  wrote:

> Here's the gist of what I would propose...
> https://gist.github.com/d3x0r/f496d0032476ed8b6f980f7ed31280da
>
> In C, there are two operators . and -> used to access members of struct
> and union types. These operators are specified such that they are always
> paired in usage; for example, if the left hand expression is a pointer to a
> struct or union, then the operator -> MUST be used. There is no occasion
> where . and -> may be interchanged, given the existing specification.
>
> It should be very evident to the compiler whether the token before '.' or
> '->' is a pointer to a struct/union or a struct/union, and just build the
> appropriate output.
>
> The source modification for the compiler is very slight, even depending on
> flag_c2x(that's not it's name).  It ends up changing a lot of existing
> lines, just to change their indentation; but that shouldn't really count
> against 'changed lines'.
>
> I'm sure, after 4 score and some years ('78-19) that it must surely have
> come up before?  Anyone able to point me to those existing proposals?
>
> D
>