Re: Transformation of contrib/check_GNU_style.sh to a python script

[Martin Liška]

Hello.

I'm sending final (slightly updated) version of the script. I'm also adding 
Jakub,
because I remember he's got another regex patterns he's using for review 
process?
Would it be fine to just remove the old *.sh script, or is it preferred to have
them both living next to each other for some time?

Thanks,
Martin
#!/usr/bin/env python3
#
# Checks some of the GNU style formatting rules in a set of patches.
#
# This file is part of GCC.
#
# GCC is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 3, or (at your option) any later
# version.
#
# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GCC; see the file COPYING3.  If not see
# .  */
#
# The script requires following python packages
# (can be installed via "pip 3 install"):
#   unidiff
#   termcolor

import sys
import re
import argparse
import unittest

from termcolor import colored
from unidiff import PatchSet
from itertools import *

ws_char = '█'
ts = 8

def error_string(s):
return colored(s, 'red', attrs = ['bold'])

class CheckError:
def __init__(self, filename, lineno, console_error, error_message,
column = -1):
self.filename = filename
self.lineno = lineno
self.console_error = console_error
self.error_message = error_message
self.column = column

def error_location(self):
return '%s:%d:%d:' % (self.filename, self.lineno,
self.column if self.column != -1 else -1)

class LineLengthCheck:
def __init__(self):
self.limit = 80
self.expanded_tab = ' ' * ts

def check(self, filename, lineno, line):
line_expanded = line.replace('\t', self.expanded_tab)
if len(line_expanded) > self.limit:
return CheckError(filename, lineno,
line_expanded[:self.limit]
+ error_string(line_expanded[self.limit:]),
'lines should not exceed 80 characters', self.limit)

return None

class SpacesCheck:
def __init__(self):
self.expanded_tab = ' ' * ts

def check(self, filename, lineno, line):
i = line.find(self.expanded_tab)
if i != -1:
return CheckError(filename, lineno,
line.replace(self.expanded_tab, error_string(ws_char * ts)),
'blocks of 8 spaces should be replaced with tabs', i)

class TrailingWhitespaceCheck:
def __init__(self):
self.re = re.compile('(\s+)$')

def check(self, filename, lineno, line):
m = self.re.search(line)
if m != None:
return CheckError(filename, lineno,
line[:m.start(1)] + error_string(ws_char * len(m.group(1)))
+ line[m.end(1):],
'trailing whitespace', m.start(1))

class SentenceSeparatorCheck:
def __init__(self):
self.re = re.compile('\w\.(\s|\s{3,})\w')

def check(self, filename, lineno, line):
m = self.re.search(line)
if m != None:
return CheckError(filename, lineno,
line[:m.start(1)] + error_string(ws_char * len(m.group(1)))
+ line[m.end(1):],
'dot, space, space, new sentence', m.start(1))

class SentenceEndOfCommentCheck:
def __init__(self):
self.re = re.compile('\w\.(\s{0,1}|\s{3,})\*/')

def check(self, filename, lineno, line):
m = self.re.search(line)
if m != None:
return CheckError(filename, lineno,
line[:m.start(1)] + error_string(ws_char * len(m.group(1)))
+ line[m.end(1):],
'dot, space, space, end of comment', m.start(1))

class SentenceDotEndCheck:
def __init__(self):
self.re = re.compile('\w(\s*\*/)')

def check(self, filename, lineno, line):
m = self.re.search(line)
if m != None:
return CheckError(filename, lineno,
line[:m.start(1)] + error_string(m.group(1)) + line[m.end(1):],
'dot, space, space, end of comment', m.start(1))

class FunctionParenthesisCheck:
# TODO: filter out GTY stuff
def __init__(self):
self.re = re.compile('\w(\s{2,})?(\()')

def check(self, filename, lineno, line):
if '#define' in line:
return None

m = self.re.search(line)
if m != None:
return CheckError(filename, lineno,
line[:m.start(2)] + error_string(m.group(2)) + line[m.end(2):],
'there should be exactly one space between function name ' \
'and parenthesis', m.start(2))

class SquareBracketCheck:
def __init__(self):
  

Re: Transformation of contrib/check_GNU_style.sh to a python script

[Tom de Vries]

On 05/19/2017 11:51 AM, Martin Liška wrote:

Hello.

I'm sending final (slightly updated) version of the script. I'm also adding 
Jakub,
because I remember he's got another regex patterns he's using for review 
process?
Would it be fine to just remove the old *.sh script, or is it preferred to have
them both living next to each other for some time?



I'd like to keep the old script around for a while, to make comparison 
between the two scripts easier.



Thanks,
Martin


check_GNU_style.py


#!/usr/bin/env python3
#
# Checks some of the GNU style formatting rules in a set of patches.
#
# This file is part of GCC.
#
# GCC is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 3, or (at your option) any later
# version.
#
# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
# WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GCC; see the file COPYING3.  If not see
# .  */
#
# The script requires following python packages
# (can be installed via "pip 3 install"):


pip3 (there's a space inbetween pip and 3)


#   unidiff
#   termcolor



I'd prefer a formulation that can be copy-pasted to the command line, f.i.:
...
# The script requires python packages, which can be installed via pip3
# like this:
# $ pip3 install unidiff termcolor
...

I'm not sure if it's possible in python, but it would be even better to 
detect the missing packages at runtime and print a message explaining 
how to install the missing packages.



Anyway, using your explanation I now managed to install the dependencies 
and run the script. [ Earlier I ran into the missing package error, 
googled the package, found https://pypi.python.org/pypi/unidiff, ran the 
suggested install line '$ pip install unidiff', and found that the 
script still was not working. ]


Thanks,
- Tom

Re: Help with rich_location and GIMPLE stmts

[Marek Polacek]

On Thu, May 18, 2017 at 01:22:02PM +0200, Martin Liška wrote:
> On 05/16/2017 09:14 PM, David Malcolm wrote:
> > On Mon, 2017-05-15 at 15:36 +0200, Martin Liška wrote:
> >> Hi.
> >>
> >> I sent this email to David some time ago, but it should be probably
> >> answered
> >> on gcc mailing list.
> > 
> >> I have idea one to improve gcov tool and I'm interested in more
> >> precise locations for gimple
> >> statements. For gcov purpose, we dump location in ipa-profile pass,
> >> which is an early IPA
> >> pass and this data is used by gcov tool to map statements (blocks) to
> >> lines of code.
> >>
> >> I did a small experiment on the place we emit the location data:
> >>  inform (gimple_location (stmt), "output_location");
> >>
> >> and it shows for:
> >> $ cat m2.c
> >> unsigned int
> >> UuT (void)
> >> { unsigned int true_var = 1; unsigned int false_var = 0; unsigned int
> >> ret = 0; if (true_var) /* count(1) */ { if (false_var) /* count(1) */
> >> ret = 111; /* count(#) */ } else ret = 999; /* count(#) */
> >> return ret; }
> >>
> >> int
> >> main (int argc, char **argv)
> >> {
> >>   UuT ();
> >>   return 0;
> >> }
> >>
> >> $ gcc --coverage m2.c
> >> m2.c: In function ‘main’:
> >> m2.c:8:3: note: output_location
> >>UuT ();
> >>^~
> >> # .MEM_2 = VDEF <.MEM_1(D)>
> >> UuT ();
> >> m2.c:9:10: note: output_location
> >>return 0;
> >>   ^
> >> _3 = 0;
> >> m2.c: In function ‘UuT’:
> >> m2.c:3:16: note: output_location
> >>  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
> >> int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
> >> count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
> >> count(#) */ return ret; }
> >> ^~~~
> >> true_var_3 = 1;
> >> m2.c:3:43: note: output_location
> >>  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
> >> int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
> >> count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
> >> count(#) */ return ret; }
> >>^
> >> false_var_4 = 0;
> >> m2.c:3:71: note: output_location
> >>  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
> >> int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
> >> count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
> >> count(#) */ return ret; }
> >>  
> >>   ^~~
> >> ret_5 = 0;
> >> m2.c:3:83: note: output_location
> >>  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
> >> int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
> >> count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
> >> count(#) */ return ret; }
> >>  
> >>   ^
> >> if (true_var_3 != 0)
> >> m2.c:3:114: note: output_location
> >>  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
> >> int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
> >> count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
> >> count(#) */ return ret; }
> >>  
> >>  ^
> >> if (false_var_4 != 0)
> >> m2.c:3:145: note: output_location
> >>  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
> >> int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
> >> count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
> >> count(#) */ return ret; }
> >>  
> >>  
> >>^
> >> ret_7 = 111;
> >> m2.c:3:182: note: output_location
> >>  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
> >> int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
> >> count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
> >> count(#) */ return ret; }
> >>  
> >>  
> >> ^
> >> ret_6 = 999;
> >> m2.c:3:215: note: output_location
> >>  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
> >> int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
> >> count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
> >> count(#) */ return ret; }
> >>  
> >>  
> >>  
> >> ^~~
> >> _8 = ret_2;
> >> m2.c:3:215: note: output_location
> >> # VUSE <.MEM_9(D)>
> >> return _8;
> >>
> >> Which is not optimal, for some assignmen

Re: Transformation of contrib/check_GNU_style.sh to a python script

[Martin Liška]

On 05/19/2017 12:39 PM, Tom de Vries wrote:
> On 05/19/2017 11:51 AM, Martin Liška wrote:
>> Hello.
>>
>> I'm sending final (slightly updated) version of the script. I'm also adding 
>> Jakub,
>> because I remember he's got another regex patterns he's using for review 
>> process?
>> Would it be fine to just remove the old *.sh script, or is it preferred to 
>> have
>> them both living next to each other for some time?
>>
> 
> I'd like to keep the old script around for a while, to make comparison 
> between the two scripts easier.

Good, thus I'm going to install the script.

> 
>> Thanks,
>> Martin
>>
>>
>> check_GNU_style.py
>>
>>
>> #!/usr/bin/env python3
>> #
>> # Checks some of the GNU style formatting rules in a set of patches.
>> #
>> # This file is part of GCC.
>> #
>> # GCC is free software; you can redistribute it and/or modify it under
>> # the terms of the GNU General Public License as published by the Free
>> # Software Foundation; either version 3, or (at your option) any later
>> # version.
>> #
>> # GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> # WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> # FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
>> # for more details.
>> #
>> # You should have received a copy of the GNU General Public License
>> # along with GCC; see the file COPYING3.  If not see
>> # .  */
>> #
>> # The script requires following python packages
>> # (can be installed via "pip 3 install"):
> 
> pip3 (there's a space inbetween pip and 3)
> 
>> #   unidiff
>> #   termcolor
>>
> 
> I'd prefer a formulation that can be copy-pasted to the command line, f.i.:
> ...
> # The script requires python packages, which can be installed via pip3
> # like this:
> # $ pip3 install unidiff termcolor
> ...

Thanks, done that.

> 
> I'm not sure if it's possible in python, but it would be even better to 
> detect the missing packages at runtime and print a message explaining how to 
> install the missing packages.

Yep, it's possible to catch ImportError and error message with command 
invocation is displayed.

> 
> 
> Anyway, using your explanation I now managed to install the dependencies and 
> run the script. [ Earlier I ran into the missing package error, googled the 
> package, found https://pypi.python.org/pypi/unidiff, ran the suggested 
> install line '$ pip install unidiff', and found that the script still was not 
> working. ]

Yep, pip installs Python2 packages.

Martin

> 
> Thanks,
> - Tom

>From 451729ac374f3721534d7965496cd8835d5122e5 Mon Sep 17 00:00:00 2001
From: marxin 
Date: Fri, 19 May 2017 15:44:23 +0200
Subject: [PATCH] Introduce new check_GNU_style.py script.

contrib/ChangeLog:

2017-05-19  Martin Liska  

	* check_GNU_style.py: New file.
---
 contrib/check_GNU_style.py | 278 +
 1 file changed, 278 insertions(+)
 create mode 100755 contrib/check_GNU_style.py

diff --git a/contrib/check_GNU_style.py b/contrib/check_GNU_style.py
new file mode 100755
index 000..b236e9312da
--- /dev/null
+++ b/contrib/check_GNU_style.py
@@ -0,0 +1,278 @@
+#!/usr/bin/env python3
+#
+# Checks some of the GNU style formatting rules in a set of patches.
+# The script is a rewritten of the same bash script and should eventually
+# replace the former script.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+#
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+# for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3.  If not see
+# .  */
+#
+# The script requires python packages, which can be installed via pip3
+# like this:
+# $ pip3 install unidiff termcolor 
+
+import sys
+import re
+import argparse
+import unittest
+
+try:
+from termcolor import colored
+except ImportError:
+print('termcolor module is missing (run: pip3 install termcolor)')
+exit(3)
+
+try:
+from unidiff import PatchSet
+except ImportError:
+print('unidiff module is missing (run: pip3 install unidiff)')
+exit(3)
+
+from itertools import *
+
+ws_char = '█'
+ts = 8
+
+def error_string(s):
+return colored(s, 'red', attrs = ['bold'])
+
+class CheckError:
+def __init__(self, filename, lineno, console_error, error_message,
+column = -1):
+self.filename = filename
+self.lineno = lineno
+self.console_error = console_error
+self.error_message = error_message
+self.column = column
+
+def error_location(self):
+return '

Re: Help with rich_location and GIMPLE stmts

[Martin Liška]

On 05/19/2017 02:14 PM, Marek Polacek wrote:
> On Thu, May 18, 2017 at 01:22:02PM +0200, Martin Liška wrote:
>> On 05/16/2017 09:14 PM, David Malcolm wrote:
>>> On Mon, 2017-05-15 at 15:36 +0200, Martin Liška wrote:
 Hi.

 I sent this email to David some time ago, but it should be probably
 answered
 on gcc mailing list.
>>>
 I have idea one to improve gcov tool and I'm interested in more
 precise locations for gimple
 statements. For gcov purpose, we dump location in ipa-profile pass,
 which is an early IPA
 pass and this data is used by gcov tool to map statements (blocks) to
 lines of code.

 I did a small experiment on the place we emit the location data:
  inform (gimple_location (stmt), "output_location");

 and it shows for:
 $ cat m2.c
 unsigned int
 UuT (void)
 { unsigned int true_var = 1; unsigned int false_var = 0; unsigned int
 ret = 0; if (true_var) /* count(1) */ { if (false_var) /* count(1) */
 ret = 111; /* count(#) */ } else ret = 999; /* count(#) */
 return ret; }

 int
 main (int argc, char **argv)
 {
   UuT ();
   return 0;
 }

 $ gcc --coverage m2.c
 m2.c: In function ‘main’:
 m2.c:8:3: note: output_location
UuT ();
^~
 # .MEM_2 = VDEF <.MEM_1(D)>
 UuT ();
 m2.c:9:10: note: output_location
return 0;
   ^
 _3 = 0;
 m2.c: In function ‘UuT’:
 m2.c:3:16: note: output_location
  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
 int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
 count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
 count(#) */ return ret; }
 ^~~~
 true_var_3 = 1;
 m2.c:3:43: note: output_location
  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
 int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
 count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
 count(#) */ return ret; }
^
 false_var_4 = 0;
 m2.c:3:71: note: output_location
  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
 int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
 count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
 count(#) */ return ret; }
  
   ^~~
 ret_5 = 0;
 m2.c:3:83: note: output_location
  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
 int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
 count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
 count(#) */ return ret; }
  
   ^
 if (true_var_3 != 0)
 m2.c:3:114: note: output_location
  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
 int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
 count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
 count(#) */ return ret; }
  
  ^
 if (false_var_4 != 0)
 m2.c:3:145: note: output_location
  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
 int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
 count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
 count(#) */ return ret; }
  
  
^
 ret_7 = 111;
 m2.c:3:182: note: output_location
  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
 int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
 count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
 count(#) */ return ret; }
  
  
 ^
 ret_6 = 999;
 m2.c:3:215: note: output_location
  { unsigned int true_var = 1; unsigned int false_var = 0; unsigned
 int ret = 0; if (true_var) /* count(1) */ { if (false_var) /*
 count(1) */ ret = 111; /* count(#) */ } else ret = 999; /*
 count(#) */ return ret; }
  
  
  
 ^~~
 _8 = ret_2;
 m2.c:3:215: note: output_location
 # VUSE <.MEM_9(D)>
 return _8;
>>>

Re: git-svn error due to out-of-sync changes?

[Jeff Law]

On 05/18/2017 01:42 PM, Martin Sebor wrote:
> On 05/18/2017 12:55 PM, Markus Trippelsdorf wrote:
>> On 2017.05.18 at 12:41 -0600, Martin Sebor wrote:
>>> On 05/18/2017 11:59 AM, Jeff Law wrote:
 On 05/18/2017 11:41 AM, Martin Sebor wrote:
> I just tried to push a change and got the error below.  git
> pull says my tree is up to date.  I wonder if it's caused by
> my commit conflicting with another commit (in this case
> r248244) that git-svn doesn't see because it lags behind SVN.
> I brushed this (and other strange errors) off before, not
> bothering to try to understand it but it's happened enough
> times that I'd like to bring it up.  I expect some (maybe
> even most) of these issues would not exist if we were using
> Git directly rather than the git-svn wrapper. Has any more
> progress been made on the Git integration project?  Is there
> something I/we can do to help get it done?
 That just means something changed upstream betwen your last git svn
 rebase and your local commit.

 Just "git svn rebase", resolve conflicts (the ChangeLogs are the most
 common source of conflicts)  and you should be good to go.
>>>
>>> The main issue is that there tend to be errors that wouldn't
>>> happen without the extra layer between Git and SVN.  The two
>>> are out of sync by minutes (I don't know exactly how many but
>>> it seems like at least 10), so clearing these things up takes
>>> time.  Some (I'd say most) of the errors I've seen out of
>>> Git-svn are also not completely intuitive so it's not always
>>> clear what or where the problem is.
>>>
>>> So I'd like to see if there's something that can be done to
>>> move the migration forward.
>>
>> The same issue also happen with git when several people push at the same
>> time.
> 
> Yes, it can.  The major difference, I suspect, is due to Git-svn
> asynchronous, delayed updates. 
I don't think so.  git-svn is going right into the SVN repository.
THere's no delayed update or anything like that that's affecting this
action.

What's happened is someone has made an upstream commit into the SVN repo
between the time you last updated your local tree via git svn rebase and
hte time you tried to push via git svn dcommit.

Moving to GIT exclusively won't really change this because the same
situation happens if you git pull, change locally, then git push.  If
someone changes the remote repo between the pull/push you get a conflict.

jeff

Re: git-svn error due to out-of-sync changes?

[Jeff Law]

On 05/18/2017 02:06 PM, Markus Trippelsdorf wrote:
> On 2017.05.18 at 13:42 -0600, Martin Sebor wrote:
>> On 05/18/2017 12:55 PM, Markus Trippelsdorf wrote:
>>> On 2017.05.18 at 12:41 -0600, Martin Sebor wrote:
 On 05/18/2017 11:59 AM, Jeff Law wrote:
> On 05/18/2017 11:41 AM, Martin Sebor wrote:
>> I just tried to push a change and got the error below.  git
>> pull says my tree is up to date.  I wonder if it's caused by
>> my commit conflicting with another commit (in this case
>> r248244) that git-svn doesn't see because it lags behind SVN.
>> I brushed this (and other strange errors) off before, not
>> bothering to try to understand it but it's happened enough
>> times that I'd like to bring it up.  I expect some (maybe
>> even most) of these issues would not exist if we were using
>> Git directly rather than the git-svn wrapper. Has any more
>> progress been made on the Git integration project?  Is there
>> something I/we can do to help get it done?
> That just means something changed upstream betwen your last git svn
> rebase and your local commit.
>
> Just "git svn rebase", resolve conflicts (the ChangeLogs are the most
> common source of conflicts)  and you should be good to go.

 The main issue is that there tend to be errors that wouldn't
 happen without the extra layer between Git and SVN.  The two
 are out of sync by minutes (I don't know exactly how many but
 it seems like at least 10), so clearing these things up takes
 time.  Some (I'd say most) of the errors I've seen out of
 Git-svn are also not completely intuitive so it's not always
 clear what or where the problem is.

 So I'd like to see if there's something that can be done to
 move the migration forward.
>>>
>>> The same issue also happen with git when several people push at the same
>>> time.
>>
>> Yes, it can.  The major difference, I suspect, is due to Git-svn
>> asynchronous, delayed updates.  My guess is that Git-svn pull
>> requests are based on updates from SVN that happen only every
>> few minutes, but pushes happen in real time.  So when we pull,
>> we're likely to get outdated sources (changes committed since
>> the last Git update are not included).  But when we push, we're
>> likely to run into (at a minimum) ChangeLog conflicts with those
>> already committed changes that Git-svn hasn't been updated with.
>> This is just a wild guess based on the errors I've seen and
>> their increased incidence since 7 has been released.
> 
> "git svn dcommit" will run "git svn rebase" automatically, so you are
> already on the latest svn revision. It doesn't matter if git lags behind
> or not, only svn counts here.
> The situation will be exactly the same after the switch to git. If you
> are unlucky and several people push at the same time, you could be
> forced to pull/rebase and retry your push request several times.
Right.

jeff

Re: Basic Block Statistics

[Jeff Law]

On 05/17/2017 08:22 PM, Will Hawkins wrote:
> On Wed, May 17, 2017 at 2:59 PM, Will Hawkins  wrote:
>> On Wed, May 17, 2017 at 2:41 PM, Will Hawkins  wrote:
>>> On Wed, May 17, 2017 at 1:04 PM, Will Hawkins  wrote:
 On Wed, May 17, 2017 at 1:02 PM, Jeff Law  wrote:
> On 05/17/2017 10:36 AM, Will Hawkins wrote:
>> As I started looking into this, it seems like PLUGIN_FINISH is where
>> my plugin will go. Everything is great so far. However, when plugins
>> at that event are invoked, they get no data. That means I will have to
>> look into global structures for information regarding the compilation.
>> Are there pointers to the documentation that describe the relevant
>> global data structures that are accessible at this point?
>>
>> I am looking through the source code and documentation and can't find
>> what I am looking for. I am happy to continue working, but thought I'd
>> ask just in case I was missing something silly.
>>
>> Thanks again for all your help getting me started on this!
> FOR_EACH_BB (bb) is what you're looking for.  That will iterate over the
> basic blocks.

 Thank you so much for your response!

 I just found this as soon as you sent it. Sorry for wasting your time!


>
> Assuming you're running late, you'll then want to walk each insn within
> the bb.  So something like this
>
> basic_block bb
> FOR_EACH_BB (bb)
>   {
> rtx_insn *insn;
> FOR_BB_INSNS (bb, insn)
>   {
> /* Do something with INSN.  */
>   }
>   }
>
>
> Note that if you're running too late the CFG may have been released, in
> which case this code wouldn't do anything.
>>>
>>> This macro seems to require that there be a valid cfun. This seems to
>>> imply that the macro will work only where the plugin callback is
>>> invoked before/after a pass that does some optimization for a
>>> particular function. In particular, at PLUGIN_FINISH, cfun is NULL.
>>> This makes perfect sense.
>>>
>>> Since PLUGIN_FINISH is the place where diagnostics are supposed to be
>>> printed, I was wondering if there was an equivalent iterator for all
>>> translation units (from which I could derive functions, from which I
>>> could derive basic blocks) that just "FINISH"ed compiling?
>>
>>
>> Answering my own question for historical purposes and anyone else who
>> might need this:
>>
>>   FOR_EACH_VEC_ELT(*all_translation_units, i, t)
>>
>> is exactly what I was looking for!
>>
>> Sorry for the earlier spam and thank you for your patience!
>> Will
> 
> 
> Well, I thought that this was what I wanted, but it turns out perhaps
> I was wrong. So, I am turning back for some help. Again, i apologize
> for the incessant emails.
> 
> I would have thought that a translation unit tree node's chain would
> point to all the nested tree nodes. This does not seem to be the case,
> however. Am I missing something? Or is this the intended behavior?
I think there's a fundamental misunderstanding.

We don't hold the RTL IR for all the functions in a translation unit in
memory at the same time.  You have to look at the RTL IR for each as its
generated.

jeff

Re: git-svn error due to out-of-sync changes?

[Jason Merrill]

On Thu, May 18, 2017 at 3:42 PM, Martin Sebor  wrote:
> On 05/18/2017 12:55 PM, Markus Trippelsdorf wrote:
>> On 2017.05.18 at 12:41 -0600, Martin Sebor wrote:
>>> On 05/18/2017 11:59 AM, Jeff Law wrote:
 On 05/18/2017 11:41 AM, Martin Sebor wrote:
>
> I just tried to push a change and got the error below.  git
> pull says my tree is up to date.  I wonder if it's caused by
> my commit conflicting with another commit (in this case
> r248244) that git-svn doesn't see because it lags behind SVN.
> I brushed this (and other strange errors) off before, not
> bothering to try to understand it but it's happened enough
> times that I'd like to bring it up.  I expect some (maybe
> even most) of these issues would not exist if we were using
> Git directly rather than the git-svn wrapper. Has any more
> progress been made on the Git integration project?  Is there
> something I/we can do to help get it done?

 That just means something changed upstream betwen your last git svn
 rebase and your local commit.

 Just "git svn rebase", resolve conflicts (the ChangeLogs are the most
 common source of conflicts)  and you should be good to go.
>>>
>>> The main issue is that there tend to be errors that wouldn't
>>> happen without the extra layer between Git and SVN.  The two
>>> are out of sync by minutes (I don't know exactly how many but
>>> it seems like at least 10), so clearing these things up takes
>>> time.  Some (I'd say most) of the errors I've seen out of
>>> Git-svn are also not completely intuitive so it's not always
>>> clear what or where the problem is.
>>>
>>> So I'd like to see if there's something that can be done to
>>> move the migration forward.
>>
>> The same issue also happen with git when several people push at the same
>> time.
>
> Yes, it can.  The major difference, I suspect, is due to Git-svn
> asynchronous, delayed updates.  My guess is that Git-svn pull
> requests are based on updates from SVN that happen only every
> few minutes, but pushes happen in real time.  So when we pull,
> we're likely to get outdated sources (changes committed since
> the last Git update are not included).  But when we push, we're
> likely to run into (at a minimum) ChangeLog conflicts with those
> already committed changes that Git-svn hasn't been updated with.
> This is just a wild guess based on the errors I've seen and
> their increased incidence since 7 has been released.

Yes, the git mirror can lag the SVN repo by a few minutes, that's why
you need to 'git svn rebase' to pull directly from SVN before a
commit.

Jason