Re: [EXT] Re: Option processing question

[Richard Biener]

On Mon, Jan 13, 2020 at 9:28 PM Gary Oblock  wrote:
>
> If I have an optimization set up to run in LTO (call it my_opt) and
> the flags -flto-partition=one, -flto options and -fmy-opt are set
> then the optimization might or might not be run depending of whether
> it can all fit in one partition.
>
> What I'm thinking is as long as it's a fatal error detectable anywhere
> upstream in the compilation to not specify -fmy-opt without 
> -flto-partition=one
> then all will be well. So how do I detect it at all and where would I put the 
> checking?

I guess you could always check the symbols used_from_other_partition
and/or in_other_partition flags?  That is, it might be perfectly possible
for some corner-cases to do the transform independently with more than one
partition if the partitioning happens to isolate things perfectly?

Richard.

> Gary
> 
> From: Richard Biener 
> Sent: Monday, January 13, 2020 2:30 AM
> To: Gary Oblock ; Jan Hubicka 
> Cc: gcc@gcc.gnu.org 
> Subject: [EXT] Re: Option processing question
>
> External Email
>
> --
> On Sat, Jan 11, 2020 at 4:47 AM Gary Oblock  wrote:
> >
> > I'm writing an LTO optimization that requires "-flto-partition=one" How can 
> > I make
> > sure that this is the case? I've spent hours greping the code and the 
> > Internals Doc is
> > worth less than nothing for something like this.
>
> That's of course because you shouldn't be doing this ;)
>
> > If you have an answer or even
> > I good idea of where to look please let me know. Note, for an normal 
> > "-fbalh" there's
> > a flag_blah that I could look at but for this there seems to be zip, nil, 
> > diddly squat, etc.
>
> At LTRANS time you indeed don't know.
>
> But there's -flto-partition=none (none, not one), that you can detect somehow
> (I think in_lto_p && !flag_ltrans && !flag_wpa).
>
> Richard.
>
> > Many thanks,
> >
> > Gary

Re: [EXT] Re: Option processing question

[Jan Hubicka]

> On Mon, Jan 13, 2020 at 9:28 PM Gary Oblock  wrote:
> >
> > If I have an optimization set up to run in LTO (call it my_opt) and
> > the flags -flto-partition=one, -flto options and -fmy-opt are set
> > then the optimization might or might not be run depending of whether
> > it can all fit in one partition.
> >
> > What I'm thinking is as long as it's a fatal error detectable anywhere
> > upstream in the compilation to not specify -fmy-opt without 
> > -flto-partition=one
> > then all will be well. So how do I detect it at all and where would I put 
> > the checking?
> 
> I guess you could always check the symbols used_from_other_partition
> and/or in_other_partition flags?  That is, it might be perfectly possible
> for some corner-cases to do the transform independently with more than one
> partition if the partitioning happens to isolate things perfectly?

Well, if you do optimization at ltrans you basically see symbols in
ohter partitions as external so there is not that much of practical
difference in the implementation of analysis except that it will catch
fewer transforms.

But the idea for keeping -flto-partition=none around was to make it
possible to write a prototype of IPA pass that does not nees to play
with partitioning and incrementally add WPA bits.
It seems that this strategy may work here - having a prototype that
works only w/o WHOPR but produces valid code & does something useful
seems like good first step (which is hard enough at its own).

So I would suggest to look at -flto-partition=none first and keep in
mind that eventually we will want a solution that plays well with
partitioning.

Honza
> 
> Richard.
> 
> > Gary
> > 
> > From: Richard Biener 
> > Sent: Monday, January 13, 2020 2:30 AM
> > To: Gary Oblock ; Jan Hubicka 
> > Cc: gcc@gcc.gnu.org 
> > Subject: [EXT] Re: Option processing question
> >
> > External Email
> >
> > --
> > On Sat, Jan 11, 2020 at 4:47 AM Gary Oblock  wrote:
> > >
> > > I'm writing an LTO optimization that requires "-flto-partition=one" How 
> > > can I make
> > > sure that this is the case? I've spent hours greping the code and the 
> > > Internals Doc is
> > > worth less than nothing for something like this.
> >
> > That's of course because you shouldn't be doing this ;)
> >
> > > If you have an answer or even
> > > I good idea of where to look please let me know. Note, for an normal 
> > > "-fbalh" there's
> > > a flag_blah that I could look at but for this there seems to be zip, nil, 
> > > diddly squat, etc.
> >
> > At LTRANS time you indeed don't know.
> >
> > But there's -flto-partition=none (none, not one), that you can detect 
> > somehow
> > (I think in_lto_p && !flag_ltrans && !flag_wpa).
> >
> > Richard.
> >
> > > Many thanks,
> > >
> > > Gary

contrib/gcc_update does not work

[Uros Bizjak]

gcc_update, when called from newly initialized and pulled tree does not work:

--cut here--
$ contrib/gcc_update
Updating GIT tree
There is no tracking information for the current branch.
Please specify which branch you want to rebase against.
See git-pull(1) for details.

git pull  

If you wish to set tracking information for this branch you can do so with:

git branch --set-upstream-to=origin/ master

Adjusting file timestamps
git pull of full tree failed.
--cut here--

I would also appreciate a simple step-by step instructions on how to
set-up the local repo and basic workflow with git for "non-power"
users, as was the case with now obsolete instructions for anonymous
SVN r/o access [1] and r/w access [2]. Basically, push-my-first-patch
example.

[1] https://gcc.gnu.org/svn.html
[2] https://gcc.gnu.org/svnwrite.html

Thanks,
Uros.

Re: git conversion in progress

[Georg-Johann Lay]

Am 11.01.20 um 02:18 schrieb Joseph Myers:

I encourage people to continue to work on improving the documentation for
using git with GCC
( and
 list some of
the things that it seems it might be useful to document).  Once the
conversion is done we'll need to update various places referring to SVN on
the website and in the manuals.


Hi, some random notes:

In order to set user.name and user.email, the doc is using --global.  At 
least for me, the latter is not what I want because I am using git in 
other contexts than contributing to GCC (and am using different e-mail 
then).


In my git setup there is the following which I found quite useful when 
changing .md, or more specifically diff'ing md's and having them reviewed.


diff files allow to indicate what has been changed, for example when you 
changed some function, in the respective hunk the function name is 
displayed.  We can achieve the same for define_insn and similar:


In .gitconfig:

[diff "md"]
  # .git/info/attributes maps *.md to diff=md
  # You can also use .gitattributes for this
  xfuncname = "^[ \t]*\\([ \t]*define_.*$"

This defines the hunk to match define_insn, define_expand etc.  The 
mapping can be performed in the files mentioned above and will read:


# Allow for custom diff hunks in your (global) .gitconfig
# e.g. for machine descriptions the respective entry could read
#
# [diff "md"]
#  # GCC/.gitattributes maps *.md to diff=md
#  xfuncname = "^[ \t]*\\([ \t]*define_.*$"

# GCC machine description
*.md  diff=md

I am not planning to propose respective changes, so if you find it 
helpful, maybe some maintainer can add it.


To see the difference, try

$ git show ec9b602c167fb7dfba717

for example.  In the case it's regarded as helpful, maybe it can added 
where it applies to all branches, not only to one specific branch. 
(Dunno what changing .git/info, .git/config actually does, presumably it 
only applies if the repo is cloned).



Sometimes, one wants a different editor for commit and merge messages 
than the default one (dunno which one this is, presumably determined by 
some env variable).


The respective entry in .gitconfig is:
[core]
editor = my-editor

or can be achieved by

git config [--global] core.editor="...".  In my case, it's a script that 
gets diffs from ChangeLog deltas and composes them to a commit message 
and then calls my preferred editor on that.


And a question wrt e-mail addresses: I have 2 addresses associated with 
my GCC contributions: one @gcc.gnu.org from my GCC account and one 
non-gnu as mentioned in MAINTAINERS.  Which one should I use for commits?
As far as I remember, the gnu address can cause some problems, but I 
don't remember where...



Johann

Whitespace at the start of first line of commit

[Jakub Jelinek]

Hi!

I've noticed that a couple of Jason's commits show up in gcc-cvs
in mutt as:
[gcc r10-5937] ?PR c++/92582 - ICE with member template as requirement.
The ? in there comes from a tab character, the full subject is like
Subject: 
=?utf-8?q?=5Bgcc_r10-5937=5D_=09PR_c++/92582_-_ICE_with_member_template_a?=
 =?utf-8?q?s_requirement=2E?=

One possibility to deal with this is:
--- hooks/updates/__init__.py   2020-01-12 22:30:37.143193572 +0100
+++ hooks/updates/__init__.py   2020-01-14 11:20:05.746749843 +0100
@@ -315,7 +315,7 @@ class AbstractUpdate(object):
 subject = '[%(repo)s%(branch)s] %(subject)s' % {
 'repo': self.email_info.project_name,
 'branch': branch,
-'subject': commit.subject[:SUBJECT_MAX_SUBJECT_CHARS],
+'subject': commit.subject[:SUBJECT_MAX_SUBJECT_CHARS].strip (),
 }
 
 # Generate the body of the email in two pieces:
(untested), another, suggested by Richard on IRC, would be to reject
commits where the first line starts with whitespace.

So, what do we want to do here?

Jakub

Re: contrib/gcc_update does not work

[Jonathan Wakely]

On Tue, 14 Jan 2020 at 09:22, Uros Bizjak  wrote:
>
> gcc_update, when called from newly initialized and pulled tree does not work:

Initialized how?
If you do a 'git clone' then it correctly checks out master and sets
it to track origin/master.

>
> --cut here--
> $ contrib/gcc_update
> Updating GIT tree
> There is no tracking information for the current branch.
> Please specify which branch you want to rebase against.
> See git-pull(1) for details.
>
> git pull  
>
> If you wish to set tracking information for this branch you can do so with:
>
> git branch --set-upstream-to=origin/ master
>
> Adjusting file timestamps
> git pull of full tree failed.
> --cut here--
>
> I would also appreciate a simple step-by step instructions on how to
> set-up the local repo and basic workflow with git for "non-power"
> users, as was the case with now obsolete instructions for anonymous
> SVN r/o access [1] and r/w access [2]. Basically, push-my-first-patch
> example.
>
> [1] https://gcc.gnu.org/svn.html
> [2] https://gcc.gnu.org/svnwrite.html

They're still a work in progress:
https://gcc.gnu.org/git.html
https://gcc.gnu.org/gitwrite.html

Re: contrib/gcc_update does not work

[Uros Bizjak]

On Tue, Jan 14, 2020 at 11:34 AM Jonathan Wakely  wrote:
>
> On Tue, 14 Jan 2020 at 09:22, Uros Bizjak  wrote:
> >
> > gcc_update, when called from newly initialized and pulled tree does not 
> > work:
>
> Initialized how?

 1035  mkdir gcc
 1036  cd gcc
 1037  git init
 1038  git pull https://gcc.gnu.org/git/gcc.git

> If you do a 'git clone' then it correctly checks out master and sets
> it to track origin/master.

I see, I'll try this now.

>
> >
> > --cut here--
> > $ contrib/gcc_update
> > Updating GIT tree
> > There is no tracking information for the current branch.
> > Please specify which branch you want to rebase against.
> > See git-pull(1) for details.
> >
> > git pull  
> >
> > If you wish to set tracking information for this branch you can do so with:
> >
> > git branch --set-upstream-to=origin/ master
> >
> > Adjusting file timestamps
> > git pull of full tree failed.
> > --cut here--
> >
> > I would also appreciate a simple step-by step instructions on how to
> > set-up the local repo and basic workflow with git for "non-power"
> > users, as was the case with now obsolete instructions for anonymous
> > SVN r/o access [1] and r/w access [2]. Basically, push-my-first-patch
> > example.
> >
> > [1] https://gcc.gnu.org/svn.html
> > [2] https://gcc.gnu.org/svnwrite.html
>
> They're still a work in progress:
> https://gcc.gnu.org/git.html
> https://gcc.gnu.org/gitwrite.html

Yes, this is the information i was looking for. Sorry for being impatient ;)

Thanks,
Uros.

Re: contrib/gcc_update does not work

[Jonathan Wakely]

On Tue, 14 Jan 2020 at 10:38, Uros Bizjak  wrote:
>
> On Tue, Jan 14, 2020 at 11:34 AM Jonathan Wakely  
> wrote:
> >
> > On Tue, 14 Jan 2020 at 09:22, Uros Bizjak  wrote:
> > >
> > > gcc_update, when called from newly initialized and pulled tree does not 
> > > work:
> >
> > Initialized how?
>
>  1035  mkdir gcc
>  1036  cd gcc
>  1037  git init
>  1038  git pull https://gcc.gnu.org/git/gcc.git

Don't do that :-)

You've created an new, empty repository and then filled it with the
content from gcc.gnu.org/git/gcc but that's not the same as making a
clone of that repo. You could make it the same, but you'd have to do a
lot more steps manually. Just use 'git clone'.

> > If you do a 'git clone' then it correctly checks out master and sets
> > it to track origin/master.
>
> I see, I'll try this now.

Re: git conversion in progress

[Georg-Johann Lay]

Am 11.01.20 um 02:18 schrieb Joseph Myers:

I encourage people to continue to work on improving the documentation for
using git with GCC
( and
 list some of
the things that it seems it might be useful to document).


Nitpick:

https://gcc.gnu.org/git.html#tags "Branches and Tags"

writes:

A branch called branchname can be checked out with the following command:

git clone -b branchname ...

Referring to this as "checking out" is confusing IMO, because it may be 
confused with

git checkout -b branchname

Whereas the latter is a checkout of some branch (or changing the branch) 
within the same local copy, the former is *cloning* into an individual, 
fresh local copy which has only the mentioned branch.


Dito for tags.

Cloning with only 1 branch is usually done because it faster downloads 
and consumes less local memory.  When one works with more than one 
branch (usually when backporting stuff etc.), the mentioned


git clone --reference original-gcc ...

might be preferable.

Johann

Re: git conversion in progress

[Richard Earnshaw]

On 14/01/2020 11:01, Georg-Johann Lay wrote:
> Am 11.01.20 um 02:18 schrieb Joseph Myers:
>> I encourage people to continue to work on improving the documentation for
>> using git with GCC
>> ( and
>>  list some of
>> the things that it seems it might be useful to document).
> 
> Nitpick:
> 
> https://gcc.gnu.org/git.html#tags "Branches and Tags"
> 
> writes:
> 
> A branch called branchname can be checked out with the following command:
> 
>     git clone -b branchname ...
> 
> Referring to this as "checking out" is confusing IMO, because it may be
> confused with
>     git checkout -b branchname
> 
> Whereas the latter is a checkout of some branch (or changing the branch)
> within the same local copy, the former is *cloning* into an individual,
> fresh local copy which has only the mentioned branch.
> 
> Dito for tags.
> 
> Cloning with only 1 branch is usually done because it faster downloads
> and consumes less local memory.  When one works with more than one
> branch (usually when backporting stuff etc.), the mentioned

Well it's likely that the server would have to repack the objects on the
fly to supply just one branch; and it does that less well than the base
pack).  So you'd probably end up with a much larger initial download
than just fetching the entire history (currently ~750M).  It would
likely take longer to fetch as well since the server has to do much more
work to generate that custom pack.

Yes, if you want to work with that single branch long-term you can
repack it again locally (if you have enough ram to produce a really
dense pack).

So frankly, I wouldn't recommend downloading just a single branch.

> 
> git clone --reference original-gcc ...
> 
> might be preferable.
> 
> Johann

Re: git conversion in progress

[Andreas Schwab]

On Jan 14 2020, Georg-Johann Lay wrote:

> or can be achieved by
>
> git config [--global] core.editor="...".

Space instead of = between key and value.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Do we want to add -fsanitize=function?

[Martin Liška]

Hi.

The missing sanitizer reports about violations of function signatures
for indirect calls, like:

$ cat sanitize-function.cpp
#include 

void f() {}
void (*fnpointer) (int);

void save () {
  fnpointer = reinterpret_cast(reinterpret_cast(f));
}

int main(void) {
  save ();
  fnpointer (32);
}

$ clang++ sanitize-function.cpp -fsanitize=function -g && ./a.out
sanitize-function.cpp:12:3: runtime error: call to function f() through pointer 
to incorrect function type 'void (*)(int)'
/home/marxin/Programming/testcases/sanitize-function.cpp:3: note: f() defined 
here
#0 0x431c57 in main 
/home/marxin/Programming/testcases/sanitize-function.cpp:12:3
#1 0x7f6284994e0a in __libc_start_main 
/usr/src/debug/glibc-2.30-2.1.x86_64/csu/../csu/libc-start.c:308:16
#2 0x403349 in _start 
/home/abuild/rpmbuild/BUILD/glibc-2.30/csu/../sysdeps/x86_64/start.S:120

The sanitizer leverages the following UBSAN API:

void __ubsan_handle_function_type_mismatch_v1(FunctionTypeMismatchData *Data,
  ValueHandle Function,
  ValueHandle calleeRTTI,
  ValueHandle fnRTTI) {
}

Which is quite obvious API, except the last argument. The last argument is a 
pointer to RTTI
of a function pointer that will be used for indirect calls. Having a pointer to 
a fn, clang
emits the following sequence at the very beginning of a function:

void save () {
  431bb0:   eb 06   jmp431bb8 <_Z4savev+0x8>
  431bb2:   76 32   jbe431be6 
  431bb4:   e8 90 00 00 55  callq  55431c49 <_end+0x546d8979>
  431bb9:   48 89 e5mov%rsp,%rbp

so it jump +8 and the content of the next 8 bytes is actually a pointer to RTTI 
of this function.
That's how can one get RTTI of a fn pointer. The checking code then verifies 
that a dereferenced
function really contains the 431bb8 jump at the very beginning of a function.
The suggested approach is very target-dependent and quite hackish.

So my question is if we want the sanitizer? And second, do we have something 
similar that does
so explicit .text emission w/o GAS assistance?

Thanks,
Martin

Re: git conversion in progress

[Georg-Johann Lay]

Am 14.01.20 um 12:34 schrieb Andreas Schwab:

On Jan 14 2020, Georg-Johann Lay wrote:


git clone --reference original-gcc ...


Don't use --reference.  It is too easy to lose work if you don't know
what you are doing.

Andreas.


Well, then it should not be proposed in git.html then?

There may be git noobs like me that follow the recommendations from there.

Johann

Re: git conversion in progress

[Andreas Schwab]

On Jan 14 2020, Georg-Johann Lay wrote:

> git clone --reference original-gcc ...

Don't use --reference.  It is too easy to lose work if you don't know
what you are doing.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: git conversion in progress

[Jakub Jelinek]

On Tue, Jan 14, 2020 at 12:34:13PM +0100, Andreas Schwab wrote:
> On Jan 14 2020, Georg-Johann Lay wrote:
> 
> > git clone --reference original-gcc ...
> 
> Don't use --reference.  It is too easy to lose work if you don't know
> what you are doing.

Wouldn't git clone --reference original-gcc --dissociate ...
be ok?  I.e. just save bandwidth when needing second repo if for whatever
reason git worktree isn't possible?

Jakub

Re: Do we want to add -fsanitize=function?

[Jakub Jelinek]

On Tue, Jan 14, 2020 at 12:36:11PM +0100, Martin Liška wrote:
> The missing sanitizer reports about violations of function signatures
> for indirect calls, like:
> 
> $ cat sanitize-function.cpp
> #include 
> 
> void f() {}
> void (*fnpointer) (int);
> 
> void save () {
>   fnpointer = reinterpret_cast(reinterpret_cast(f));
> }
> 
> int main(void) {
>   save ();
>   fnpointer (32);
> }

_Z4savev:   # @_Z4savev
.cfi_startproc
.long   846595819   # 0x327606eb
.long   .L__unnamed_2-_Z4savev
# %bb.0:# %entry
...
seems to be what they emit on x86_64.  Now, wonder what they do on other
targets, and how does it play with all the other options that add stuff
to the start of functions, e.g. -fcf-protection=full (where it needs to
really start with endbr64 instruction), or the various options for
patcheable function entries, -mfentry, profiling and the like.

Jakub

Re: contrib/gcc_update does not work

[Richard Biener]

On Tue, Jan 14, 2020 at 11:44 AM Jonathan Wakely  wrote:
>
> On Tue, 14 Jan 2020 at 10:38, Uros Bizjak  wrote:
> >
> > On Tue, Jan 14, 2020 at 11:34 AM Jonathan Wakely  
> > wrote:
> > >
> > > On Tue, 14 Jan 2020 at 09:22, Uros Bizjak  wrote:
> > > >
> > > > gcc_update, when called from newly initialized and pulled tree does not 
> > > > work:
> > >
> > > Initialized how?
> >
> >  1035  mkdir gcc
> >  1036  cd gcc
> >  1037  git init
> >  1038  git pull https://gcc.gnu.org/git/gcc.git
>
> Don't do that :-)

I think it's what the wiki still suggests...

> You've created an new, empty repository and then filled it with the
> content from gcc.gnu.org/git/gcc but that's not the same as making a
> clone of that repo. You could make it the same, but you'd have to do a
> lot more steps manually. Just use 'git clone'.
>
> > > If you do a 'git clone' then it correctly checks out master and sets
> > > it to track origin/master.
> >
> > I see, I'll try this now.

Re: contrib/gcc_update does not work

[Andrew Pinski]

On Tue, Jan 14, 2020 at 4:10 AM Richard Biener
 wrote:
>
> On Tue, Jan 14, 2020 at 11:44 AM Jonathan Wakely  
> wrote:
> >
> > On Tue, 14 Jan 2020 at 10:38, Uros Bizjak  wrote:
> > >
> > > On Tue, Jan 14, 2020 at 11:34 AM Jonathan Wakely  
> > > wrote:
> > > >
> > > > On Tue, 14 Jan 2020 at 09:22, Uros Bizjak  wrote:
> > > > >
> > > > > gcc_update, when called from newly initialized and pulled tree does 
> > > > > not work:
> > > >
> > > > Initialized how?
> > >
> > >  1035  mkdir gcc
> > >  1036  cd gcc
> > >  1037  git init
> > >  1038  git pull https://gcc.gnu.org/git/gcc.git
> >
> > Don't do that :-)
>
> I think it's what the wiki still suggests...

Yes the wiki does suggest that but the wiki was about using git mirror.
"With a normal git repository 'git clone' grabs all branches, but
because of the SVN mirror we need to do something more elaborate" :).
This is no longer true and even wrong as the branches are in a
different location really.

Thanks,
Andrew

>
> > You've created an new, empty repository and then filled it with the
> > content from gcc.gnu.org/git/gcc but that's not the same as making a
> > clone of that repo. You could make it the same, but you'd have to do a
> > lot more steps manually. Just use 'git clone'.
> >
> > > > If you do a 'git clone' then it correctly checks out master and sets
> > > > it to track origin/master.
> > >
> > > I see, I'll try this now.

Re: git conversion in progress

[Andreas Schwab]

On Jan 14 2020, Jakub Jelinek wrote:

> On Tue, Jan 14, 2020 at 12:34:13PM +0100, Andreas Schwab wrote:
>> On Jan 14 2020, Georg-Johann Lay wrote:
>> 
>> > git clone --reference original-gcc ...
>> 
>> Don't use --reference.  It is too easy to lose work if you don't know
>> what you are doing.
>
> Wouldn't git clone --reference original-gcc --dissociate ...
> be ok?  I.e. just save bandwidth when needing second repo if for whatever
> reason git worktree isn't possible?

Yes, this is safe, since the new clone will be independent and
standalone.  It will contain a duplicate of the object database, though.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: contrib/gcc_update does not work

[Andreas Schwab]

On Jan 14 2020, Andrew Pinski wrote:

> On Tue, Jan 14, 2020 at 4:10 AM Richard Biener
>  wrote:
>>
>> On Tue, Jan 14, 2020 at 11:44 AM Jonathan Wakely  
>> wrote:
>> >
>> > On Tue, 14 Jan 2020 at 10:38, Uros Bizjak  wrote:
>> > >
>> > > On Tue, Jan 14, 2020 at 11:34 AM Jonathan Wakely  
>> > > wrote:
>> > > >
>> > > > On Tue, 14 Jan 2020 at 09:22, Uros Bizjak  wrote:
>> > > > >
>> > > > > gcc_update, when called from newly initialized and pulled tree does 
>> > > > > not work:
>> > > >
>> > > > Initialized how?
>> > >
>> > >  1035  mkdir gcc
>> > >  1036  cd gcc
>> > >  1037  git init
>> > >  1038  git pull https://gcc.gnu.org/git/gcc.git
>> >
>> > Don't do that :-)
>>
>> I think it's what the wiki still suggests...
>
> Yes the wiki does suggest that but the wiki was about using git mirror.

Even with the old mirror a normal clone was working as usual.  Only if
you wanted to commit using git svn you needed to do the unusual setup.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Do we want to add -fsanitize=function?

[Martin Liška]

On 1/14/20 12:54 PM, Jakub Jelinek wrote:

On Tue, Jan 14, 2020 at 12:36:11PM +0100, Martin Liška wrote:

The missing sanitizer reports about violations of function signatures
for indirect calls, like:

$ cat sanitize-function.cpp
#include 

void f() {}
void (*fnpointer) (int);

void save () {
   fnpointer = reinterpret_cast(reinterpret_cast(f));
}

int main(void) {
   save ();
   fnpointer (32);
}


_Z4savev:   # @_Z4savev
.cfi_startproc
.long   846595819   # 0x327606eb
.long   .L__unnamed_2-_Z4savev
# %bb.0:# %entry
...
seems to be what they emit on x86_64.  Now, wonder what they do on other
targets


Other targets are not supported :P


, and how does it play with all the other options that add stuff
to the start of functions, e.g. -fcf-protection=full (where it needs to
really start with endbr64 instruction)


Using the options one will get:

_Z4savev:   # @_Z4savev
.cfi_startproc
.long   846595819   # 0x327606eb
.long   .L__unnamed_2-_Z4savev
# %bb.0:
endbr64

So endbr64 is placed after the RTTI record.


, or the various options for
patcheable function entries, -mfentry, profiling and the like.


These work similarly, then follow the RTTI record:

_Z4savev:   # @_Z4savev
.cfi_startproc
.long   846595819   # 0x327606eb
.long   .L__unnamed_2-_Z4savev
# %bb.0:
callq   __fentry__

Martin



Jakub

Re: Do we want to add -fsanitize=function?

[Jakub Jelinek]

On Tue, Jan 14, 2020 at 01:57:47PM +0100, Martin Liška wrote:
> > seems to be what they emit on x86_64.  Now, wonder what they do on other
> > targets
> 
> Other targets are not supported :P
> 
> > , and how does it play with all the other options that add stuff
> > to the start of functions, e.g. -fcf-protection=full (where it needs to
> > really start with endbr64 instruction)
> 
> Using the options one will get:
> 
> _Z4savev:   # @_Z4savev
>   .cfi_startproc
>   .long   846595819   # 0x327606eb
>   .long   .L__unnamed_2-_Z4savev
> # %bb.0:
>   endbr64
> 
> So endbr64 is placed after the RTTI record.

Which is wrong, this will then fail on CET hardware.

Jakub

Re: git conversion in progress

[Jonathan Wakely]

On Tue, 14 Jan 2020 at 11:37, Georg-Johann Lay  wrote:
>
> Am 14.01.20 um 12:34 schrieb Andreas Schwab:
> > On Jan 14 2020, Georg-Johann Lay wrote:
> >
> >> git clone --reference original-gcc ...
> >
> > Don't use --reference.  It is too easy to lose work if you don't know
> > what you are doing.
> >
> > Andreas.
>
> Well, then it should not be proposed in git.html then?

It's a work in progress. I've already suggested that worktrees are a
better solution for people who want to work on multiple branches but
save disk space.

Re: contrib/gcc_update does not work

[Jonathan Wakely]

On Tue, 14 Jan 2020 at 12:46, Andreas Schwab  wrote:
>
> On Jan 14 2020, Andrew Pinski wrote:
>
> > On Tue, Jan 14, 2020 at 4:10 AM Richard Biener
> >  wrote:
> >>
> >> On Tue, Jan 14, 2020 at 11:44 AM Jonathan Wakely  
> >> wrote:
> >> >
> >> > On Tue, 14 Jan 2020 at 10:38, Uros Bizjak  wrote:
> >> > >
> >> > > On Tue, Jan 14, 2020 at 11:34 AM Jonathan Wakely 
> >> > >  wrote:
> >> > > >
> >> > > > On Tue, 14 Jan 2020 at 09:22, Uros Bizjak  wrote:
> >> > > > >
> >> > > > > gcc_update, when called from newly initialized and pulled tree 
> >> > > > > does not work:
> >> > > >
> >> > > > Initialized how?
> >> > >
> >> > >  1035  mkdir gcc
> >> > >  1036  cd gcc
> >> > >  1037  git init
> >> > >  1038  git pull https://gcc.gnu.org/git/gcc.git
> >> >
> >> > Don't do that :-)
> >>
> >> I think it's what the wiki still suggests...
> >
> > Yes the wiki does suggest that but the wiki was about using git mirror.
>
> Even with the old mirror a normal clone was working as usual.  Only if
> you wanted to commit using git svn you needed to do the unusual setup.

I've added a big bold notice to the top of the wiki page:
The information on this page refers to the git-svn mirror WHICH HAS
BEEN RETIRED. While some tips here may still be useful (like the
convenient git aliases) do not follow these steps to use the new Git
repository (which is not a mirror, so obviously this page isn't
talking about it!)

Re: git conversion in progress

[Andrew Stubbs]

On 14/01/2020 13:00, Jonathan Wakely wrote:

On Tue, 14 Jan 2020 at 11:37, Georg-Johann Lay  wrote:


Am 14.01.20 um 12:34 schrieb Andreas Schwab:

On Jan 14 2020, Georg-Johann Lay wrote:


git clone --reference original-gcc ...


Don't use --reference.  It is too easy to lose work if you don't know
what you are doing.

Andreas.


Well, then it should not be proposed in git.html then?


It's a work in progress. I've already suggested that worktrees are a
better solution for people who want to work on multiple branches but
save disk space.


Worktrees are better, for most purposes, but they still have the same 
risks as reference repos "I don't need that old tree any more."


Both are best left for the advanced tricks section.

Andrew

Re: Do we want to add -fsanitize=function?

[Martin Liška]

On 1/14/20 1:59 PM, Jakub Jelinek wrote:

On Tue, Jan 14, 2020 at 01:57:47PM +0100, Martin Liška wrote:

seems to be what they emit on x86_64.  Now, wonder what they do on other
targets


Other targets are not supported :P


, and how does it play with all the other options that add stuff
to the start of functions, e.g. -fcf-protection=full (where it needs to
really start with endbr64 instruction)


Using the options one will get:

_Z4savev:   # @_Z4savev
.cfi_startproc
.long   846595819   # 0x327606eb
.long   .L__unnamed_2-_Z4savev
# %bb.0:
endbr64

So endbr64 is placed after the RTTI record.


Which is wrong, this will then fail on CET hardware.


Sure, which is a minor limitation. FCF is supposed to be production
security feature while UBSAN is more for a testing playground.

Martin



Jakub

Re: Do we want to add -fsanitize=function?

[Jakub Jelinek]

On Tue, Jan 14, 2020 at 02:56:38PM +0100, Martin Liška wrote:
> On 1/14/20 1:59 PM, Jakub Jelinek wrote:
> > On Tue, Jan 14, 2020 at 01:57:47PM +0100, Martin Liška wrote:
> > > > seems to be what they emit on x86_64.  Now, wonder what they do on other
> > > > targets
> > > 
> > > Other targets are not supported :P
> > > 
> > > > , and how does it play with all the other options that add stuff
> > > > to the start of functions, e.g. -fcf-protection=full (where it needs to
> > > > really start with endbr64 instruction)
> > > 
> > > Using the options one will get:
> > > 
> > > _Z4savev:   # @_Z4savev
> > >   .cfi_startproc
> > >   .long   846595819   # 0x327606eb
> > >   .long   .L__unnamed_2-_Z4savev
> > > # %bb.0:
> > >   endbr64
> > > 
> > > So endbr64 is placed after the RTTI record.
> > 
> > Which is wrong, this will then fail on CET hardware.
> 
> Sure, which is a minor limitation. FCF is supposed to be production
> security feature while UBSAN is more for a testing playground.

But then the compiler should just fail if you mix the two, rather than
emitting something that doesn't work at all.
Or better fix the design, so that it can grok an endbr64 together with
the following jump as another magic.

Jakub

Re: git conversion in progress

[Martin Jambor]

Hi,

On Tue, Jan 14 2020, Andreas Schwab wrote:
> On Jan 14 2020, Georg-Johann Lay wrote:
>
>> git clone --reference original-gcc ...
>
> Don't use --reference.  It is too easy to lose work if you don't know
> what you are doing.

What are the risks, assuming I won't delete the referenced repo which
sits on the same partition of the same local disk as the new one?

I prefer it to worktree because I often just do have master checked out
in multiple directories and worktree would be unhappy about that... and
the risk of accidentally deleting the base worktree are IMHO the same as
the risk of deleting the referenced repo.

Anything that I am missing?  (In other words, do I know what I am doing? :-)

Thanks,

Martin

Re: git conversion in progress

[Andreas Schwab]

On Jan 14 2020, Martin Jambor wrote:

> Hi,
>
> On Tue, Jan 14 2020, Andreas Schwab wrote:
>> On Jan 14 2020, Georg-Johann Lay wrote:
>>
>>> git clone --reference original-gcc ...
>>
>> Don't use --reference.  It is too easy to lose work if you don't know
>> what you are doing.
>
> What are the risks, assuming I won't delete the referenced repo which
> sits on the same partition of the same local disk as the new one?

The risk is if the original repository is gc'd (and nowadays git
automatically runs git gc --auto from time to time) it may lose objects
that are still needed by the referencing repository.  That doesn't
happen with git worktree as the main repository knows about all
references, including worktree local reflogs.

> I prefer it to worktree because I often just do have master checked out
> in multiple directories and worktree would be unhappy about that... and
> the risk of accidentally deleting the base worktree are IMHO the same as
> the risk of deleting the referenced repo.

I'd suggest making more use of local branches.  They are cheep and easy
to use with git.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: git conversion in progress

[Jonathan Wakely]

On Tue, 14 Jan 2020 at 14:28, Martin Jambor  wrote:
>
> Hi,
>
> On Tue, Jan 14 2020, Andreas Schwab wrote:
> > On Jan 14 2020, Georg-Johann Lay wrote:
> >
> >> git clone --reference original-gcc ...
> >
> > Don't use --reference.  It is too easy to lose work if you don't know
> > what you are doing.
>
> What are the risks, assuming I won't delete the referenced repo which
> sits on the same partition of the same local disk as the new one?

If I understand correctly (and I don't use --reference so I might be
wrong), with --reference you need to do a git fetch in each separate
repo when you want to upate, because although they share objects they
each have their own "view" of the remotes.

If you do a git pull in a clone created with --reference it will fetch
new objects and store them locally, it won't update the referenced
repo. This means they can diverge over time and aren't sharing as much
as possible (I assume that can be resolved by repack or gc? I don't
know).

Also, anything committed to any worktree is instantly visible to the
others. I have worktrees for master, gcc-9 and gcc-8. After I backport
something to gcc-9 (but before I push it upstream) I can go to the
gcc-8 worktree and do 'git cherry-pick gcc-9' to apply the newest
commit from gcc-9 branch to the gcc-8 branch. There's no need to push
or pull anything between the worktrees, because they are all using the
same clone with the same local branches.

> I prefer it to worktree because I often just do have master checked out
> in multiple directories and worktree would be unhappy about that...

Sounds like a good use for branches :-)

> and
> the risk of accidentally deleting the base worktree are IMHO the same as
> the risk of deleting the referenced repo.

For me, there is zero chance that I'll ever think ~/src/gcc/gcc is an
old tree I don't use. It's also the one where I do work on 'master'
(or branches created from 'master') so I'm unlikely to ever think
that's an unwanted throwaway tree that I don't need. With worktrees
you'll only have 'master' checked out in one of your trees, so it
doesn't take a lot of discipline to know which one is the one
containing all the .git/objects and other repository metadata.

> Anything that I am missing?  (In other words, do I know what I am doing? :-)

Re: Do we want to add -fsanitize=function?

[Martin Liška]

On 1/14/20 3:00 PM, Jakub Jelinek wrote:

But then the compiler should just fail if you mix the two, rather than
emitting something that doesn't work at all.
Or better fix the design, so that it can grok an endbr64 together with
the following jump as another magic.


Sure. One can make an error when these 2 options are mixed together.
So the question still remains opened, do we want to implement the
sanitizer feature?

Martin

Re: Do we want to add -fsanitize=function?

[Jakub Jelinek]

On Tue, Jan 14, 2020 at 04:15:54PM +0100, Martin Liška wrote:
> On 1/14/20 3:00 PM, Jakub Jelinek wrote:
> > But then the compiler should just fail if you mix the two, rather than
> > emitting something that doesn't work at all.
> > Or better fix the design, so that it can grok an endbr64 together with
> > the following jump as another magic.
> 
> Sure. One can make an error when these 2 options are mixed together.
> So the question still remains opened, do we want to implement the
> sanitizer feature?

IMHO not for GCC 10, for GCC 11, it really sounds too hackish, so unsure.
It should at least cover more than one arch and have these issues like CET
etc. discussed upstream.

Jakub

Towards removal of gcc/DATESTAMP

[Jakub Jelinek]

Hi!

The following command prints the same string as DATESTAMP file
contains in all gcc-7 and later based branches I've tried so far (and nothing 
when
e.g. invoked from within svn checkout).

o=$(git config --get gcc-config.upstream); test -z "$o" && o=origin; r=$(cat 
BASE-VER | cut -d. -f 1); b=; if git rev-parse --verify --quiet 
$o/releases/gcc-$r >/dev/null; then b=origin/releases/gcc-$r; elif git 
rev-parse --verify --quiet $o/releases/gcc-9 >/dev/null; then b=$o/master; fi; 
test -n "$b" && TZ=UTC LC_ALL=C git log --date=iso -1 $(git merge-base HEAD $b) 
| sed -n 's/^Date:[[:blank:]]*//p' | sed 's/ .*$//;s/-//g'

I haven't bothered with the bumping at 00:16 UTC, the script assumes bump
on 00:00 UTC instead.

Do we want to replace DATESTAMP bumping with something like that?

There are several issues that need to be figured out:

1) some trees will not be git repositories, e.g. release tarballs, snapshot
   tarballs or e.g. vendor gcc tarball snapshots; for release
   tarballs/snapshots, we could arrange for gcc_release to create
   gcc/DATESTAMP file that way and have some script e.g. for vendors to
   use that would do the same; gcc/Makefile.in then could use
   gcc/DATESTAMP if it is present, fallback to git command above (perhaps
   made more portable if needed) and fail if neither works?
2) in gcc/Makefile.in, e.g. version.o depends on DATESTAMP file and the
   rebuild happens when the timestamp on that file changes; should we
   e.g. create gcc/DATESTAMP file in the build directory with some
   move-if-change hacks around and let version.o depend on a goal that
   will do that and on the build directory gcc/DATESTAMP?
3) should we handle gcc/REVISION stuff similarly (and use what
   gcc-descr alias does without that alias) for that commit, perhaps
   with --full or maybe just with somewhat smaller --abbrev= (say 20 rather
   than 40 chars)?

Jakub

Help with new GCC git workflow...

[Peter Bergner]

As somewhat of a git newbie and given gcc developers will do a git push of
our changes rather than employing a git pull development model, I'd like
a little hand holding on what my new gcc git workflow should be, so I don't
screw up the upstream repo by pushing something to the wrong place. :-)

I know enough that I should be using local branches to develop my changes,
so I want something like:

  git checkout master
  git pull
  git checkout -b 
  
  git commit -m "My commit message1"
  
  git commit -m "My commit message2"
  
  git commit -m "My commit message3"
  

At this point, I get a little confused. :-)  I know to submit my patch
for review, I'll want to squash my commits down into one patch, but how
does one do that?  Should I do that now or only when I'm ready to
push this change to the upstream repo or ???  Do I need to even do that?

Also, when I'm ready to push this "change" upstream to trunk, I'll need
to move this over to my master and then push.  What are the recommended
commands for doing that?  I assume I need to rebase my branch to
current upstream master, since that probably has moved forward since
I checked my code out.

Also, at what point do I write my final commit message, which is different
than the (possibly simple) commit messages above?  Is that done after I've
pulled my local branch into my master?  ...or before?  ...or during the
merge over?

...and this is just for changes going to trunk.  How does all this change
when I want to push changes to a release or vendor branch?

I guess I'm just looking for some simple workflow commands for both
trunk and release/vendor branches I can follow until I'm a little more
confident in my git knowledge.

I'm guessing I'm not the only one who would like this info, so maybe
someone can add this to our wiki?


Peter

Re: git conversion in progress

[Joseph Myers]

On Tue, 14 Jan 2020, Georg-Johann Lay wrote:

> In order to set user.name and user.email, the doc is using --global.  At least
> for me, the latter is not what I want because I am using git in other contexts
> than contributing to GCC (and am using different e-mail then).

Using --global is the simple case for someone who hasn't used git before 
on the system where they are committing changes to GCC (the page says "If 
you have not already configured those in $HOME/.gitconfig).  It gives 
instructions for the case of using a different email address with GCC - 
after giving instructions for the simplest case of a new git user.

> In .gitconfig:
> 
> [diff "md"]
>   # .git/info/attributes maps *.md to diff=md
>   # You can also use .gitattributes for this
>   xfuncname = "^[ \t]*\\([ \t]*define_.*$"

That's already in contrib/gcc-git-customization.sh - but we need to 
document that script in git.html, and also need to add the relevant line 
to the checked-in .gitattributes file.

> And a question wrt e-mail addresses: I have 2 addresses associated with my GCC
> contributions: one @gcc.gnu.org from my GCC account and one non-gnu as
> mentioned in MAINTAINERS.  Which one should I use for commits?
> As far as I remember, the gnu address can cause some problems, but I don't
> remember where...

You can use whatever address you prefer for committing to GCC.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Whitespace at the start of first line of commit

[Joseph Myers]

On Tue, 14 Jan 2020, Jakub Jelinek wrote:

> (untested), another, suggested by Richard on IRC, would be to reject
> commits where the first line starts with whitespace.

I'd suggest making the hooks reject whitespace at the start of the first 
line of the commit message.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: git conversion in progress

[Joseph Myers]

On Tue, 14 Jan 2020, Georg-Johann Lay wrote:

> A branch called branchname can be checked out with the following command:
> 
> git clone -b branchname ...
> 
> Referring to this as "checking out" is confusing IMO, because it may be
> confused with
> git checkout -b branchname
> 
> Whereas the latter is a checkout of some branch (or changing the branch)
> within the same local copy, the former is *cloning* into an individual, fresh
> local copy which has only the mentioned branch.

I think the simple case for git beginners is having separate, independent 
directories for each branch being used, but I see it's been changed to 
describe checking out a branch in an existing clone instead.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: git conversion in progress

[Joseph Myers]

On Tue, 14 Jan 2020, Richard Earnshaw wrote:

> Well it's likely that the server would have to repack the objects on the
> fly to supply just one branch; and it does that less well than the base
> pack).  So you'd probably end up with a much larger initial download
> than just fetching the entire history (currently ~750M).  It would
> likely take longer to fetch as well since the server has to do much more
> work to generate that custom pack.

If you use clone with -b, it still fetches all the branches and tags that 
are fetched by default and puts the branches in remotes/origin/, it just 
changes which branch gets a corresponding branch set up in refs/heads/ and 
checked out initially.  So there is no extra repacking involved.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Towards removal of gcc/DATESTAMP

[Jakub Jelinek]

On Tue, Jan 14, 2020 at 04:52:12PM +0100, Jakub Jelinek wrote:
> The following command prints the same string as DATESTAMP file
> contains in all gcc-7 and later based branches I've tried so far (and nothing 
> when
> e.g. invoked from within svn checkout).

Jonathan wondered on IRC about the weirdo hardcoded check for releases/gcc-9.
The intent was to punt if it isn't a git tree at all, or e.g. somebody
unpacks gcc tarball without gcc/DATESTAMP file in it into his git tracked
home directory.

Perhaps better might be start with
if git merge-base --is-ancestor 633c65dda889eb887fb6ce3b04cefdeb4a69b0b3 HEAD; 
then ... ; fi
to verify it is GCC 8+ (for older we of course would never omit
gcc/DATESTAMP).

> o=$(git config --get gcc-config.upstream); test -z "$o" && o=origin; r=$(cat 
> BASE-VER | cut -d. -f 1); b=; if git rev-parse --verify --quiet 
> $o/releases/gcc-$r >/dev/null; then b=origin/releases/gcc-$r; elif git 
> rev-parse --verify --quiet $o/releases/gcc-9 >/dev/null; then b=$o/master; 
> fi; test -n "$b" && TZ=UTC LC_ALL=C git log --date=iso -1 $(git merge-base 
> HEAD $b) | sed -n 's/^Date:[[:blank:]]*//p' | sed 's/ .*$//;s/-//g'

Jakub

Re: Towards removal of gcc/DATESTAMP

[Joseph Myers]

On Tue, 14 Jan 2020, Jakub Jelinek wrote:

> o=$(git config --get gcc-config.upstream); test -z "$o" && o=origin; r=$(cat 
> BASE-VER | cut -d. -f 1); b=; if git rev-parse --verify --quiet 
> $o/releases/gcc-$r >/dev/null; then b=origin/releases/gcc-$r; elif git 
> rev-parse --verify --quiet $o/releases/gcc-9 >/dev/null; then b=$o/master; 
> fi; test -n "$b" && TZ=UTC LC_ALL=C git log --date=iso -1 $(git merge-base 
> HEAD $b) | sed -n 's/^Date:[[:blank:]]*//p' | sed 's/ .*$//;s/-//g'

That looks like it would use the author date, when the committer date is 
more likely to be monotonic and closer to what's relevant here.  
(Actually I think you want --date=iso-local --pretty=format:%cd or similar 
to print just the committer date, in the UTC timezone that you've set 
rather than the committer's timezone which is the default for printing the 
date.)

> 1) some trees will not be git repositories, e.g. release tarballs, snapshot
>tarballs or e.g. vendor gcc tarball snapshots; for release

Or they may be git repositories that don't have origin/releases/gcc-9 (for 
example) - a vendor repository with its own branch structure, say.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Help with new GCC git workflow...

[Eric S. Raymond]

Peter Bergner :
> At this point, I get a little confused. :-)  I know to submit my patch
> for review, I'll want to squash my commits down into one patch, but how
> does one do that?  Should I do that now or only when I'm ready to
> push this change to the upstream repo or ???  Do I need to even do that?

If you want to squash a commit series, the magic is git rebase -i. You
give that a number of commits to look back at at and you'll get a buffer
instructing you how to squash and shuffle that series.  You'll also be able
to edit the commit message.

I like to write really fine-grained commits when I'm developing, then
squash before pushing so the public repo commits always go from "tests
pass" to "test pass".  That way you can do clean bisections on the
public history.

> Also, when I'm ready to push this "change" upstream to trunk, I'll need
> to move this over to my master and then push.  What are the recommended
> commands for doing that?

There are a couple of ways.  I usually squash as described above
then use "git cherry-pick".  But that's because I have philosophical
reasons to avoid long-lives branches.

>   I assume I need to rebase my branch to
> current upstream master, since that probably has moved forward since
> I checked my code out.

Yes, in general you'll want to do that.

> Also, at what point do I write my final commit message, which is different
> than the (possibly simple) commit messages above?  Is that done after I've
> pulled my local branch into my master?  ...or before?  ...or during the
> merge over?

I do it at rebase -i time along with the squash of the series.
-- 
http://www.catb.org/~esr/";>Eric S. Raymond

Re: Help with new GCC git workflow...

[Jonathan Wakely]

On 14/01/20 10:07 -0600, Peter Bergner wrote:

As somewhat of a git newbie and given gcc developers will do a git push of
our changes rather than employing a git pull development model, I'd like
a little hand holding on what my new gcc git workflow should be, so I don't
screw up the upstream repo by pushing something to the wrong place. :-)

I know enough that I should be using local branches to develop my changes,
so I want something like:

 git checkout master
 git pull
 git checkout -b 
 
 git commit -m "My commit message1"
 
 git commit -m "My commit message2"
 
 git commit -m "My commit message3"
 

At this point, I get a little confused. :-)  I know to submit my patch
for review, I'll want to squash my commits down into one patch, but how
does one do that?


This is Git, there are a hundred ways ;-)



Should I do that now or only when I'm ready to
push this change to the upstream repo or ???


Totally up to you. You might want to squash some commits early, e.g.
to fix silly typos, but keep most of the branch history intact until
the last minute (to help you remember what you changed and why).
That's my preference.


Do I need to even do that?


If it's a long-lived feature branch you might want to keep the
separate commits and merge them all to master preserving the branch
history (don't take my word for it, I can't remember what we decided
should be the policy for such long-lived branches).

If it's just a short-lived branch to change one thing, or fix one bug,
then what you push should be a single, self-contained commit (even if
you happened to develop it as a series of mini-commits locally).


Also, when I'm ready to push this "change" upstream to trunk, I'll need
to move this over to my master and then push.


Strictly speaking, you don't need to. You can push that branch
directly to master:  git push origin HEAD:master
That will fail unless the current branch is up-to-date with master,
but would work fine if you've already rebased your branch on master,
or if master hasn't moved since you branched.


What are the recommended
commands for doing that?  I assume I need to rebase my branch to
current upstream master, since that probably has moved forward since
I checked my code out.


You can either rebase on the new master (i.e. bring the new stuff from
master into your branch) or the other way around (bring the stuff from
your branch into master).

A pretty straightforward way to do the second way is:

git checkout master
git pull
git merge --squash 
[resolve any merge conflicts]
[build + test again if your branch was behind master]
git push

i.e. pull the changes from your branch onto master, then push.

This leaves your  branch untouched, so you still have
all the history locally for future reference.

There are other ways e.g. 'git rebase --interactive master' and
squash all the commits in the branch that way. Interactive rebases are
awesome, and very useful. It's a fairly manual process, but that gives
you full control. Get familiar with it.

Or, to (non-interactively) rebase your branch on master (which you
might want to do periodically anyway, before you're ready to push
upstream):

git checkout master
git pull
git checkout 
git rebase master
[resolve any merge conflicts]
[build + test]

That's rebased your branch, but not actually squashed the branch
commits yet. You can do that by checking out master and doing a
merge --squash (as above) or just on the branch:

# make this branch's committed state (aka "index") the same as master
git reset master
# but that didn't touch the content of the working directory,
# that still matches your branch, so you can add+commit all the
# files in the working dir that differ from the "index":
git add --all
git commit

This alters your branch's history to be a single commit against
master, which contains all the changes that you'd done on the branch.
I've never used this method, so I hesitate to recommend it. It's the
least obvious way IMO.

And after either of the rebase methods, you still need to get that
single commit onto master to push (unless you're going to push
directly from the branch). So that might be a reason to prefer doing a
"merge --squash" to squash at the same time as pulling the changes
into master (i.e. the first method above).


Also, at what point do I write my final commit message, which is different
than the (possibly simple) commit messages above?  Is that done after I've
pulled my local branch into my master?  ...or before?  ...or during the
merge over?


Doesn't matter. As long as you push a single commit with a good commit
message, it doesn't matter when that was written. Personally I think
doing it at the end when you do the merge --squash to add a single
commit to master.


...and this is just for changes going to trunk.  How does all this change
when I want to push changes to a release or vendor branch?


It's pretty similar. Create a branch from the release branch, merge it
back to the release branch.

Personally, I don't us

Re: Help with new GCC git workflow...

[Julien "FrnchFrgg" Rivaud]

Le 14/01/2020 à 17:07, Peter Bergner a écrit :

As somewhat of a git newbie and given gcc developers will do a git push of
our changes rather than employing a git pull development model, I'd like
a little hand holding on what my new gcc git workflow should be, so I don't
screw up the upstream repo by pushing something to the wrong place. :-)

I know enough that I should be using local branches to develop my changes,
so I want something like:

   git checkout master
   git pull
   git checkout -b 
   
   git commit -m "My commit message1"
   
   git commit -m "My commit message2"
   
   git commit -m "My commit message3"
   

At this point, I get a little confused. :-)  I know to submit my patch
for review, I'll want to squash my commits down into one patch, but how
does one do that?


Whether you really want to squash your commits into one patch really 
depends on their nature. If all of the commits are really part of the 
same unit of meaning and you just committed regularly as a "timely 
backup" procedure, then yes you want to squash. In that case


git rebase -i master

is your friend: it will open an editor with the list of all your commits 
and commands applied to them (by default, "pick", that is replay the 
diff without change apart from conflict automatic resolution when 
possible and manual intervention if not). You can reorder the commits, 
and/or squash some of them together.


If the commits are actually steps to attaining a goal where each step is 
valid on its own, then you can IMHO submit the set of patches to the ML. 
An example of a set with 17 patches on the git ML is:


https://public-inbox.org/git/cover.1577185374.git.liu.den...@gmail.com/T/#m4f20ec1d987a7d3b10a0f4fa0ebb7c64acf75320

(Note that the first mail, labeled PATCH 00/17, will probably end up 
being the commit message of the merge commit, because git prefers real 
merges, even if rebased, to fast-forward merges. For GCC that wouldn't 
be so with the current policy AFAIU)


If you think that the change is big enough to warrant such patch series, 
and the commits you made as you went are not that, then again


git rebase -i master

is your friend, using more "edit" steps, or even branch again from 
master and use git "cherry-pick" and manual rewriting to build up a new 
set, comparing regularly with the end result of . A lot 
of work for sure, but worth it in the end.



Also, when I'm ready to push this "change" upstream to trunk, I'll need
to move this over to my master and then push.


You don't need to move anything over to your master. You can use "git 
format-patch" and/or "git send-email" to have git convert your branch to 
a series of mails to send for review to the ML (most submissions to the 
git ML are done that way).


When the patch set is rewiewed and Acked-By the module 
owners/maintainers, either they push themselves (using "git am" to 
convert the mails to a local branch, then push), or if you have the 
rights you do it yourself.


To do so, you need to first

git checkout master
git pull
git checkout 
git rebase master

If the rebase was not completely trivial, I guess you should submit your 
changed patches again to the ML (this is subject to appreciation by the 
gcc community of course).


If the rebase was successful, you can then push (which will *update 
master*):


git push origin :master

If you just want to push to your user namespace, then you don't 
necessarily need to rebase (though I recommend doing so regularly), and 
you should do


git push origin 
:refs/users//


(check if I got the namespacing right of course)

There are "git config" settings for pushing that can help you 
automatically push any branch to a branch with the same name in the 
refs/users// namespace when you say


git push

without any argument, but I'm not going to explain those here. I hope 
they will be soon in the documentation about how to set up your git 
repository.



...and this is just for changes going to trunk.  How does all this change
when I want to push changes to a release or vendor branch?


For a vendor branch, you'd

git push origin :refs/vendors//

Again, defining a remote other than origin, that already map to the 
correct namespace, would be better. In git you can have as many 
"upstream" repositories as you want, and even several of them can be the 
exact same one with slightly different push or fetch configuration.


_FrnchFrgg_

Re: Help with new GCC git workflow...

[Jason Merrill]

On Tue, Jan 14, 2020 at 12:12 PM Jonathan Wakely  wrote:

> On 14/01/20 10:07 -0600, Peter Bergner wrote:
> >As somewhat of a git newbie and given gcc developers will do a git push of
> >our changes rather than employing a git pull development model, I'd like
> >a little hand holding on what my new gcc git workflow should be, so I
> don't
> >screw up the upstream repo by pushing something to the wrong place. :-)
> >
> >I know enough that I should be using local branches to develop my changes,
> >so I want something like:
> >
> >  git checkout master
> >  git pull
> >  git checkout -b 
> >  
> >  git commit -m "My commit message1"
> >  
> >  git commit -m "My commit message2"
> >  
> >  git commit -m "My commit message3"
> >  
> >
> >At this point, I get a little confused. :-)  I know to submit my patch
> >for review, I'll want to squash my commits down into one patch, but how
> >does one do that?
>
> This is Git, there are a hundred ways ;-)
>
>
> >Should I do that now or only when I'm ready to
> >push this change to the upstream repo or ???
>
> Totally up to you. You might want to squash some commits early, e.g.
> to fix silly typos, but keep most of the branch history intact until
> the last minute (to help you remember what you changed and why).
> That's my preference.
>
> >Do I need to even do that?
>
> If it's a long-lived feature branch you might want to keep the
> separate commits and merge them all to master preserving the branch
> history (don't take my word for it, I can't remember what we decided
> should be the policy for such long-lived branches).
>

I think we're prohibiting merges to master.  We definitely don't want
merges of branches with commits that don't each satisfy the normal rules
for commits.

If it's just a short-lived branch to change one thing, or fix one bug,
> then what you push should be a single, self-contained commit (even if
> you happened to develop it as a series of mini-commits locally).
>
> >Also, when I'm ready to push this "change" upstream to trunk, I'll need
> >to move this over to my master and then push.
>
> Strictly speaking, you don't need to. You can push that branch
> directly to master:  git push origin HEAD:master
> That will fail unless the current branch is up-to-date with master,
> but would work fine if you've already rebased your branch on master,
> or if master hasn't moved since you branched.
>
> >What are the recommended
> >commands for doing that?  I assume I need to rebase my branch to
> >current upstream master, since that probably has moved forward since
> >I checked my code out.
>
> You can either rebase on the new master (i.e. bring the new stuff from
> master into your branch) or the other way around (bring the stuff from
> your branch into master).
>
> A pretty straightforward way to do the second way is:
>
> git checkout master
> git pull
> git merge --squash 
> [resolve any merge conflicts]
> [build + test again if your branch was behind master]
> git push
>
> i.e. pull the changes from your branch onto master, then push.
>
> This leaves your  branch untouched, so you still have
> all the history locally for future reference.
>
> There are other ways e.g. 'git rebase --interactive master' and
> squash all the commits in the branch that way. Interactive rebases are
> awesome, and very useful. It's a fairly manual process, but that gives
> you full control. Get familiar with it.
>
> Or, to (non-interactively) rebase your branch on master (which you
> might want to do periodically anyway, before you're ready to push
> upstream):
>
> git checkout master
> git pull
> git checkout 
> git rebase master
> [resolve any merge conflicts]
> [build + test]
>
> That's rebased your branch, but not actually squashed the branch
> commits yet. You can do that by checking out master and doing a
> merge --squash (as above) or just on the branch:
>
> # make this branch's committed state (aka "index") the same as master
> git reset master
> # but that didn't touch the content of the working directory,
> # that still matches your branch, so you can add+commit all the
> # files in the working dir that differ from the "index":
> git add --all
> git commit
>

If you use git reset --soft you don't need to add again.  I use this alias
all the time to combine commits:

sq = "!f(){ git reset --soft ${@:-HEAD^} && git commit --amend -C
HEAD; }; f"

this combines all the commits since into the argument commit (or the
penultimate commit, if not specifie).


> This alters your branch's history to be a single commit against
> master, which contains all the changes that you'd done on the branch.
> I've never used this method, so I hesitate to recommend it. It's the
> least obvious way IMO.
>
> And after either of the rebase methods, you still need to get that
> single commit onto master to push (unless you're going to push
> directly from the branch). So that might be a reason to prefer doing a
> "merge --squash" to squash at the same time as pulling the changes
> int

Re: Help with new GCC git workflow...

[Joseph Myers]

On Tue, 14 Jan 2020, Jason Merrill wrote:

> I think we're prohibiting merges to master.  We definitely don't want
> merges of branches with commits that don't each satisfy the normal rules
> for commits.

Yes.  The hooks prevent pushing a merge commit to master or a release 
branch.  A branch can still end up as multiple commits on master, if there 
are multiple logically separate changes there (in which case it *should* 
end up as multiple commits, one for each change that makes sense on its 
own), but each of the commits that ends up on master should be correct in 
itself (as far as known at the time they are pushed), to support 
bisectability.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: Help with new GCC git workflow...

[Joseph Myers]

On Tue, 14 Jan 2020, Jonathan Wakely wrote:

> > ...and this is just for changes going to trunk.  How does all this change
> > when I want to push changes to a release or vendor branch?
> 
> It's pretty similar. Create a branch from the release branch, merge it
> back to the release branch.
> 
> Personally, I don't usually use branches for backports, because I only
> have one in flight at a time. So I'll just make the change directly in
> the release branch and push it directly from there. I rarely have

And the git approach for backports is "git cherry-pick" (though you can 
always use the approach of just applying a patch to a different branch if 
you want).  I did some recently with "git cherry-pick -x -n -e" to try out 
the process of cherry-picking with git in GCC.  -x is useful to mark the 
commit as a cherry-pick and -n is useful to allow editing before 
committing (for ChangeLogs, in particular); not sure if -e actually does 
anything given -n.  You'll want to have set up git-merge-changelog in any 
clone you're doing such cherry-picking in, to avoid getting conflicts from 
the cherry-picking of ChangeLogs (but even without conflicts you may want 
to edit the ChangeLog entry before commit, to mark it as a backport).  If 
you're cherry-picking a commit from the SVN era, you also need to edit the 
From-SVN: line out of the commit message (the hooks will reject the push 
if you don't, because it would confuse things to have such a line in a 
commit that isn't actually one from the repository conversion process).

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: git conversion in progress

[Jason Merrill]

On Tue, Jan 14, 2020 at 9:56 AM Andreas Schwab 
wrote:

> On Jan 14 2020, Martin Jambor wrote:
>


> > On Tue, Jan 14 2020, Andreas Schwab wrote:
> >> On Jan 14 2020, Georg-Johann Lay wrote:
> >>
> >>> git clone --reference original-gcc ...
> >>
> >> Don't use --reference.  It is too easy to lose work if you don't know
> >> what you are doing.
> >
> > What are the risks, assuming I won't delete the referenced repo which
> > sits on the same partition of the same local disk as the new one?
>
> The risk is if the original repository is gc'd (and nowadays git
> automatically runs git gc --auto from time to time) it may lose objects
> that are still needed by the referencing repository.  That doesn't
> happen with git worktree as the main repository knows about all
> references, including worktree local reflogs.
>

Exactly.  I used --reference with my local copy of the old git mirror, and
it's hopelessly corrupt now due to needed objects getting gc'd from the
reference repository.  Very much not for use by novices without
--dissociate.  As the clone man page says,

   NOTE: this is a possibly dangerous operation; do not use it
unless
   you understand what it does. If you clone your repository using
   this option and then delete branches (or use any other Git
command
   that makes any existing commit unreferenced) in the source
   repository, some objects may become unreferenced (or dangling).
   These objects may be removed by normal Git operations (such as
git
   commit) which automatically call git gc --auto. (See git-gc(1).)
If
   these objects are removed and were referenced by the cloned
   repository, then the cloned repository will become corrupt.

I notice that git.html on the website doesn't match what's currently in
wwwdocs git, is automatic updating broken?

Jason

Re: git conversion in progress

[Joseph Myers]

On Tue, 14 Jan 2020, Jason Merrill wrote:

> I notice that git.html on the website doesn't match what's currently in
> wwwdocs git, is automatic updating broken?

/www/gcc/wwwdocs-checkout/cgi-bin/gcc-gitref.cgi had local changes 
(committed, but not reverted in that checkout before they were committed), 
thereby breaking automatic updates:

error: Your local changes to the following files would be overwritten by merge:
cgi-bin/gcc-gitref.cgi
Please commit your changes or stash them before you merge.
Aborting

I've now reverted that and updated that checkout and am running 
update_web_docs_git which preprocesses the whole website among other 
things.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: git conversion in progress

[Jakub Jelinek]

On Tue, Jan 14, 2020 at 06:06:36PM +, Joseph Myers wrote:
> On Tue, 14 Jan 2020, Jason Merrill wrote:
> 
> > I notice that git.html on the website doesn't match what's currently in
> > wwwdocs git, is automatic updating broken?
> 
> /www/gcc/wwwdocs-checkout/cgi-bin/gcc-gitref.cgi had local changes 
> (committed, but not reverted in that checkout before they were committed), 
> thereby breaking automatic updates:
> 
> error: Your local changes to the following files would be overwritten by 
> merge:
> cgi-bin/gcc-gitref.cgi
> Please commit your changes or stash them before you merge.
> Aborting
> 
> I've now reverted that and updated that checkout and am running 
> update_web_docs_git which preprocesses the whole website among other 
> things.

Oops, sorry for that, forgot to remove those after I was done with testing
it.

Jakub

Re: git conversion in progress

[Martin Jambor]

Hi,

On Tue, Jan 14 2020, Andreas Schwab wrote:
> On Jan 14 2020, Martin Jambor wrote:
>
>> Hi,
>>
>> On Tue, Jan 14 2020, Andreas Schwab wrote:
>>> On Jan 14 2020, Georg-Johann Lay wrote:
>>>
 git clone --reference original-gcc ...
>>>
>>> Don't use --reference.  It is too easy to lose work if you don't know
>>> what you are doing.
>>
>> What are the risks, assuming I won't delete the referenced repo which
>> sits on the same partition of the same local disk as the new one?
>
> The risk is if the original repository is gc'd (and nowadays git
> automatically runs git gc --auto from time to time) it may lose objects
> that are still needed by the referencing repository.  That doesn't
> happen with git worktree as the main repository knows about all
> references, including worktree local reflogs.

Aha, that makes sense.  In my intended setup, that *probably* would not
happen because the referenced repo would only hold stuff from the
upstream one but the danger sounds real enough for me to reconsider and
use worktrees and a number of branches pointing to the same commit as
master.

Thanks,

Martin

Re: Need sanity check on DSE vs expander issue

[Jeff Law]

On Fri, 2019-12-20 at 12:08 +0100, Richard Biener wrote:
> On December 20, 2019 8:25:18 AM GMT+01:00, Jeff Law  wrote:
> > On Fri, 2019-12-20 at 08:09 +0100, Richard Biener wrote:
> > > On December 20, 2019 3:20:40 AM GMT+01:00, Jeff Law 
> > wrote:
> > > > I need a sanity check here.
> > > > 
> > > > Given this code:
> > > > 
> > > > > typedef union { long double value; unsigned int word[4]; }
> > > > memory_long_double;
> > > > > static unsigned int ored_words[4];
> > > > > static void add_to_ored_words (long double x)
> > > > > {
> > > > >   memory_long_double m;
> > > > >   size_t i;
> > > > >   memset (&m, 0, sizeof (m));
> > > > >   m.value = x;
> > > > >   for (i = 0; i < 4; i++)
> > > > > {
> > > > >   ored_words[i] |= m.word[i];
> > > > > }
> > > > > }
> > > > > 
> > > > 
> > > > DSE is removing the memset as it thinks the assignment to m.value
> > is
> > > > going to set the entire union.
> > > > 
> > > > But when we translate that into RTL we use XFmode:
> > > > 
> > > > > ;; m.value ={v} x_6(D);
> > > > > 
> > > > > (insn 7 6 0 (set (mem/v/j/c:XF (plus:DI (reg/f:DI 77
> > > > virtual-stack-vars)
> > > > > (const_int -16 [0xfff0])) [2
> > m.value+0
> > > > S16 A128])
> > > > > (reg/v:XF 86 [ x ])) "j.c":13:11 -1
> > > > >  (nil))
> > > > > 
> > > > 
> > > > That (of course) only writes 80 bits of data because of XFmode,
> > leaving
> > > > 48 bits uninitialized.  We then read those bits, or-ing the
> > > > uninitialized data into ored_words and all hell breaks loose later.
> > > > 
> > > > Am I losing my mind?  ISTM that dse and the expander have to agree
> > on
> > > > how much data is written by the store to m.value.
> > > 
> > > It looks like MEM_SIZE is wrong here, so you need to figure how we
> > arrive at this (I guess TYPE_SIZE vs. MODE_SIZE mismatch is biting us
> > here?)
> > > That is, either the MEM should have BLKmode or the mode size should
> > match
> > > MEM_SIZE. Maybe DSE can avoid looking at MEM_SIZE for non-BLKmode
> > MEMs? 
> > It's gimple DSE that removes the memset, so it shouldn't be mucking
> > around with modes at all.  stmt_kills_ref_p seems to think the
> > assignment to m.value sets all of m.
> > 
> > The ao_ref for memset looks reasonable:
> > 
> > > (gdb) p *ref
> > > $14 = {ref = 0x0, base = 0x77ffbea0, offset = { > long>> = {coeffs = {0}}, }, 
> > >   size = {> = {coeffs = {128}},  > fields>}, max_size = {> = {
> > >   coeffs = {128}}, }, ref_alias_set = 0,
> > base_alias_set = 0, volatile_p = false}
> > 128 bits with a base of VAR_DECL m.
> > 
> > We looking to see if this statement will kill the ref:
> > 
> > > (gdb) p debug_gimple_stmt (stmt)
> > > # .MEM_8 = VDEF <.MEM_6>
> > > m.value ={v} x_7(D);
> > > $21 = void
> > > (gdb) p debug_tree (lhs)
> > >   > > type  > volatile XF
> > > size 
> > > unit-size 
> > > align:128 warn_if_not_align:0 symtab:0 alias-set -1
> > canonical-type 0x7fffea988690 precision:80>
> > > side-effects volatile
> > > arg:0  > > type  > sizes-gimplified volatile type_0 BLK size  > 128> unit-size 
> > > align:128 warn_if_not_align:0 symtab:0 alias-set -1
> > canonical-type 0x7fffea988348 fields 
> > context 
> > > pointer_to_this >
> > > side-effects addressable volatile used read BLK j.c:10:31
> > size  unit-size  > 0x7fffea7f3d38 16>
> > > align:128 warn_if_not_align:0 context  > 0x7fffea97bd00 add_to_ored_words>
> > > chain  > 0x7fffea9430a8 size_t>
> > > used unsigned read DI j.c:11:10
> > > size 
> > > unit-size 
> > > align:64 warn_if_not_align:0 context  > 0x7fffea97bd00 add_to_ored_words>>>
> > > arg:1  > > type  > XF size  unit-size  > 0x7fffea7f3d38 16>
> > > align:128 warn_if_not_align:0 symtab:0 alias-set -1
> > canonical-type 0x7fffea8133f0 precision:80
> > > pointer_to_this >
> > > XF j.c:6:29 size  unit-size
> > 
> > > align:128 warn_if_not_align:0 offset_align 128
> > > offset 
> > > bit-offset  context
> > 
> > > chain  > 0x7fffea981f18>
> > > TI j.c:6:49 size 
> > unit-size 
> > > align:32 warn_if_not_align:0 offset_align 128 offset
> >  bit-offset  > 0> context >>
> > > j.c:13:4 start: j.c:13:3 finish: j.c:13:9>
> > > $22 = void
> > > 
> > 
> > stmt_kills_ref_p calls get_ref_base_and_extent on that LHS object.  THe
> > returned base is the same as ref->base.  The returned offset is zero
> > with size/max_size of 128 bits.  So according to
> > get_ref_base_and_extent the assignment is going to write 128 bits and
> > thus kills  the memset.
> > 
> > One might argue that's where the problems start -- somewhere in
> > get_ref_base_and_extent.  
> > 
> > I'm largely offline the next couple weeks...
> > 
> > I don't have any "real" failures I'm tracking because of this, but it
> > does cause some configure generated tests to

Re: Help with new GCC git workflow...

[Jonathan Wakely]

On 14/01/20 17:05 +, Jonathan Wakely wrote:

On 14/01/20 10:07 -0600, Peter Bergner wrote:

As somewhat of a git newbie and given gcc developers will do a git push of
our changes rather than employing a git pull development model, I'd like
a little hand holding on what my new gcc git workflow should be, so I don't
screw up the upstream repo by pushing something to the wrong place. :-)

I know enough that I should be using local branches to develop my changes,
so I want something like:

git checkout master
git pull
git checkout -b 

git commit -m "My commit message1"

git commit -m "My commit message2"

git commit -m "My commit message3"


At this point, I get a little confused. :-)  I know to submit my patch
for review, I'll want to squash my commits down into one patch, but how
does one do that?


This is Git, there are a hundred ways ;-)



Should I do that now or only when I'm ready to
push this change to the upstream repo or ???


Totally up to you. You might want to squash some commits early, e.g.
to fix silly typos, but keep most of the branch history intact until
the last minute (to help you remember what you changed and why).
That's my preference.


Do I need to even do that?


If it's a long-lived feature branch you might want to keep the
separate commits and merge them all to master preserving the branch
history (don't take my word for it, I can't remember what we decided
should be the policy for such long-lived branches).

If it's just a short-lived branch to change one thing, or fix one bug,
then what you push should be a single, self-contained commit (even if
you happened to develop it as a series of mini-commits locally).


Also, when I'm ready to push this "change" upstream to trunk, I'll need
to move this over to my master and then push.


Strictly speaking, you don't need to. You can push that branch
directly to master:  git push origin HEAD:master
That will fail unless the current branch is up-to-date with master,
but would work fine if you've already rebased your branch on master,
or if master hasn't moved since you branched.


What are the recommended
commands for doing that?  I assume I need to rebase my branch to
current upstream master, since that probably has moved forward since
I checked my code out.


You can either rebase on the new master (i.e. bring the new stuff from
master into your branch) or the other way around (bring the stuff from
your branch into master).

A pretty straightforward way to do the second way is:

git checkout master
git pull
git merge --squash 
[resolve any merge conflicts]


I forgot to say that you'll need to do 'git commit' after the merge,
whether or not there were conflicts to resolve.

When you merge with --squash it adds the squashed result to the
"index" (i.e. staging area for changes to be committed) but stops
before the actual commit.

When you do the commit Git will start the commit message with all the
individual commit messages from the branch:

Squashed commit of the following:
  
commit d308da36d957feb3736be2754d134926992b3b74

Author: Jonathan Wakely 
Date:   Tue Jan 14 22:32:45 2020 +

3rd commit message

commit 3d4a8783ba7f6d466d1729b59436a96b67ddf516
Author: Jonathan Wakely 
Date:   Tue Jan 14 22:32:40 2020 +

2nd commit message

commit e0a27b98135936d4129876babdbe81e22e6e9bbf
Author: Jonathan Wakely 
Date:   Tue Jan 14 22:32:34 2020 +

1st commit message

So that's a good time to produce the final commit message, cutting and
pasting bits from those if needed, or just deleting those lines.


However, on IRC some people are saying that the simple workflow we
should be advising for newbies is to rebase and squash on the branch
(maybe using Jason's 'git reset' command) and then cherry-pick that
onto master (instead of squash-merging). A cherry-pick takes a single
commit from another branch and applies it to the current branch,
roughly equivalent to creating a patch and then manually applying it.


[build + test again if your branch was behind master]
git push

i.e. pull the changes from your branch onto master, then push.

This leaves your  branch untouched, so you still have
all the history locally for future reference.

Re: Help with new GCC git workflow...

[Jason Merrill]

On 1/14/20 5:45 PM, Jonathan Wakely wrote:

On 14/01/20 17:05 +, Jonathan Wakely wrote:

On 14/01/20 10:07 -0600, Peter Bergner wrote:
As somewhat of a git newbie and given gcc developers will do a git 
push of

our changes rather than employing a git pull development model, I'd like
a little hand holding on what my new gcc git workflow should be, so I 
don't

screw up the upstream repo by pushing something to the wrong place. :-)

I know enough that I should be using local branches to develop my 
changes,

so I want something like:

git checkout master
git pull
git checkout -b 

git commit -m "My commit message1"

git commit -m "My commit message2"

git commit -m "My commit message3"


At this point, I get a little confused. :-)  I know to submit my patch
for review, I'll want to squash my commits down into one patch, but how
does one do that?


This is Git, there are a hundred ways ;-)



Should I do that now or only when I'm ready to
push this change to the upstream repo or ???


Totally up to you. You might want to squash some commits early, e.g.
to fix silly typos, but keep most of the branch history intact until
the last minute (to help you remember what you changed and why).
That's my preference.


Do I need to even do that?


If it's a long-lived feature branch you might want to keep the
separate commits and merge them all to master preserving the branch
history (don't take my word for it, I can't remember what we decided
should be the policy for such long-lived branches).

If it's just a short-lived branch to change one thing, or fix one bug,
then what you push should be a single, self-contained commit (even if
you happened to develop it as a series of mini-commits locally).


Also, when I'm ready to push this "change" upstream to trunk, I'll need
to move this over to my master and then push.


Strictly speaking, you don't need to. You can push that branch
directly to master:  git push origin HEAD:master
That will fail unless the current branch is up-to-date with master,
but would work fine if you've already rebased your branch on master,
or if master hasn't moved since you branched.


What are the recommended
commands for doing that?  I assume I need to rebase my branch to
current upstream master, since that probably has moved forward since
I checked my code out.


You can either rebase on the new master (i.e. bring the new stuff from
master into your branch) or the other way around (bring the stuff from
your branch into master).

A pretty straightforward way to do the second way is:

git checkout master
git pull
git merge --squash 
[resolve any merge conflicts]


I forgot to say that you'll need to do 'git commit' after the merge,
whether or not there were conflicts to resolve.

When you merge with --squash it adds the squashed result to the
"index" (i.e. staging area for changes to be committed) but stops
before the actual commit.

When you do the commit Git will start the commit message with all the
individual commit messages from the branch:

     Squashed commit of the following:
     commit d308da36d957feb3736be2754d134926992b3b74
     Author: Jonathan Wakely 
     Date:   Tue Jan 14 22:32:45 2020 +

     3rd commit message

     commit 3d4a8783ba7f6d466d1729b59436a96b67ddf516
     Author: Jonathan Wakely 
     Date:   Tue Jan 14 22:32:40 2020 +

     2nd commit message

     commit e0a27b98135936d4129876babdbe81e22e6e9bbf
     Author: Jonathan Wakely 
     Date:   Tue Jan 14 22:32:34 2020 +

     1st commit message

So that's a good time to produce the final commit message, cutting and
pasting bits from those if needed, or just deleting those lines.


However, on IRC some people are saying that the simple workflow we
should be advising for newbies is to rebase and squash on the branch
(maybe using Jason's 'git reset' command)


Probably easiest to focus on rebase -i.  And perhaps git add -p for 
splitting a commit.



and then cherry-pick that
onto master (instead of squash-merging). A cherry-pick takes a single
commit from another branch and applies it to the current branch,
roughly equivalent to creating a patch and then manually applying it.


If the branch has been reshaped to be all master-ready commits, you can 
push the branch directly, or rebase your local master on top of it 
rather than cherry-pick.


Jason

Silly GIT related question

[Gary Oblock]

If you just do a clone and don't checkout a branch, is this equivalent
the  top of the trunk in the old scheme? If not then how do I get the
top of trunk?

Thanks for your patience,

Gary Oblock

Re: Silly GIT related question

[Alan Modra]

On Wed, Jan 15, 2020 at 03:11:13AM +, Gary Oblock wrote:
> If you just do a clone and don't checkout a branch, is this equivalent
> the  top of the trunk in the old scheme?

Yes.  More details in "git help clone".

-- 
Alan Modra
Australia Development Lab, IBM