Re: [lldb-dev] Inquiry regarding AddOneMoreFrame function in UnWindLLDB

2016-06-01 Thread Ravitheja Addepally via lldb-dev
Ok , currently the problem that I am facing is that there are cases in
which eh_frame should have been used for frame 0 but it isn't and the
assembly unwind just gives wrong information which could only be detected
if the debugger tried to extract more frames. Now the usage of
AddOneMoreFrame in UnwindLLDB is to try to get more than one frames in the
stack. I want to run both the unwinders and select the one that gives more
number of frames.

On Wed, Jun 1, 2016 at 12:27 AM, Jason Molenda  wrote:

>
> > On May 31, 2016, at 11:31 AM, jing...@apple.com wrote:
> >
> >
> >> On May 31, 2016, at 12:52 AM, Ravitheja Addepally via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >>
> >> Hello,
> >>  I posted this query a while ago, i still have no answers, I am
> currently working on Bug 27687 (PrintStackTraces), so the reason for the
> failure is the erroneous unwinding of the frames from the zeroth frame. The
> error is not detected in AddOneMoreFrame, since it only checks for 2 more
> frames, if it was checking more frames in AddOneMoreFrame, it would have
> detected the error. Now my questions are ->
> >>
> >> ->  is that is there any specific reason for only checking 2 frames
> instead of more ?
> >
> > The stepping machinery uses the unwinder on each stop to figure out
> whether it has stepped in or out, which is fairly performance sensitive, so
> we don't want AddOneMoreFrame to do more work than it has to.
>
>
> Most common case for a bad unwind, where the unwinder is stuck in a loop,
> is a single stack frame repeating.  I've seen loops as much as six frames
> repeating (which are not actually a series of recursive calls) but it's
> less common.
>
> >
> >> ->  Why no make the EH CFI based unwinder the default one and make the
> assembly the fallback ?
>
>
> Sources of unwind information fall into two categories.  They can describe
> the unwind state at every instruction of a function (asynchronous) or they
> can describe the unwind state only at function call boundaries
> (synchronous).
>
> Think of "asynchronous" here as the fact that the debugger can interrupt
> the program at any point in time.
>
> Most unwind information is designed for exception handling -- it is
> synchronous, it can only throw an exception in the body of the function, or
> an exception is passed up through it when it is calling another function.
>
> For exception handling, there is no need/requirement to describe the
> prologue or epilogue instructions, for instance.
>
> eh_frame (and DWARF's debug_frame from which it derives) splits the
> difference and makes things quite unclear.  It is guaranteed to be correct
> for exception handling -- it is synchronous, and is valid in the middle of
> the function and when it is calling other functions -- but it is a general
> format that CAN be asynchronous if the emitter includes information about
> the prologue or epilogue or mid-function stack changes.  But eh_frame is
> not guaranteed to be that way, and in fact there's no way for it to
> indicate what it describes, beyond the required unwind info for exception
> handling.
>
> On x86, gcc and clang have always described the prologue unwind info in
> their eh_frame.  gcc has recently started describing the epilogue too
> (clang does not).  There's code in lldb (e.g.
> UnwindAssembly_x86::AugmentUnwindPlanFromCallSite) written by Tong Shen
> when interning at Google which will try to detect if the eh_frame describes
> the prologue and epilogue.  If it does, it will use eh_frame for frame 0.
> If it only describes the prologue, it will use the instruction emulation
> code to add epilogue instructions and use that at frame 0.
>
>
> There are other sources of unwind information similar to eh_frame that are
> only for exception handling.  Tamas added ArmUnwindInfo last year which
> reads the .ARM.exidx unwind tables.  I added compact unwind importing - an
> Apple specific format that uses a single 4-byte word to describe the unwind
> state for each function, which can't describe anything in the
> prologue/epilogue.  These formats definitely can't be used to unwind at
> frame 0 because we could be stopped anywhere in the prologue/epilogue where
> they are not accurate.
>
>
> It's unfortunate that eh_frame doesn't include a way for the producer to
> declare how async the unwind info is, it makes the debugger's job a lot
> more difficult.
>
>
> J
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [cfe-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Renato Golin via lldb-dev
On 1 June 2016 at 05:16, Douglas Gregor  wrote:
> Performance can also be an issue; it takes a bunch of fast bots to keep up 
> with developers testing their pull requests, especially when what you’re 
> testing is a very large C++ code base. That said, “test and merge on success” 
> workflows are *wonderful* for keeping the buildbots happy.

Indeed. These pre-commit machines have to be dedicated, and we may
have to have more than one, depending on the volume.

But the good news is that they can scale independently, and they'll
remove a huge strain from the current buildbots. Also, I think it's
easier to justify (commercially) a few additional pre-commit bots than
duplicating every single configuration I have today.

cheers,
--renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


[lldb-dev] [Bug 27963] New: static variables are not listed with "(lldb) target variable" when initialized in-place

2016-06-01 Thread via lldb-dev
https://llvm.org/bugs/show_bug.cgi?id=27963

Bug ID: 27963
   Summary: static variables are not listed with "(lldb) target
variable" when initialized in-place
   Product: lldb
   Version: 3.8
  Hardware: PC
OS: Linux
Status: NEW
  Severity: normal
  Priority: P
 Component: All Bugs
  Assignee: lldb-dev@lists.llvm.org
  Reporter: manstet...@iue.tuwien.ac.at
CC: llvm-b...@lists.llvm.org
Classification: Unclassified

"(lldb) target variable" does not list static data members that are initialized
in-place, e.g.:

class A {
static const int i = 7;
};


When initialized like this:

class A {
static const int i;
};

const int A::i = 7;

, it works fine.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread David Chisnall via lldb-dev
On 1 Jun 2016, at 17:02, John Criswell via llvm-dev  
wrote:
> 
> Regarding the issue of git sub-modules and keeping Clang/LLVM in sync, 
> perhaps we should just put Clang and LLVM into a single git repository and 
> add a CMake option to disable compilation of Clang (the same could be done 
> for other LLVM sub-projects for which bisection and other nifty features 
> require a single revision number to refer to code across projects).  Keeping 
> these projects in separate repositories is just more work, and I don't see 
> what we're getting out of that extra effort.

The thing that we’d gain by keeping them together is not breaking all of the 
people who have forks of the clang git repo.  GitHub tracks 563 so far.  

David

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread John Criswell via lldb-dev

Dear Renato,

Do you have a set of volunteers lined up to do such a migration? Getting 
people willing to do the migration will obviously be key, and that was 
the one thing I didn't see in the original email.


Regarding the issue of git sub-modules and keeping Clang/LLVM in sync, 
perhaps we should just put Clang and LLVM into a single git repository 
and add a CMake option to disable compilation of Clang (the same could 
be done for other LLVM sub-projects for which bisection and other nifty 
features require a single revision number to refer to code across 
projects).  Keeping these projects in separate repositories is just more 
work, and I don't see what we're getting out of that extra effort.


IIRC, Clang was initially separate from LLVM because it significantly 
added to the download time and compilation time.  I don't think download 
time is really a problem anymore, and compilation time can be fixed with 
a CMake option.


For what it's worth,

John Criswell

On 5/31/16 2:31 PM, Renato Golin via cfe-dev wrote:

Folks,

There has been some discussion on IRC about SVN hosting and the perils
of doing it ourselves. The consensus on the current discussion was
that moving to a Git-only solution would have some disvantages, but
many advantages. Furthermore, not hosting our own repos would save us
a lot of headaches, admin costs and timed out connections.

TL;DR: GitHub + git submodules [1] could replace all the functionality
we have currently with SVN.

(also GitLab, BitBucketc, etc).

Here are some of the arguments made on IRC...

1. Due to SVN, we can't re-write history. If we use some GitHub
properties [2], we could have the same effect.

2. Due to SVN, we have a mandatory time sequence, so commits go first
in LLVM, then Clang (for example), and buildbots don't get lost. If we
use submodules [1], we can have a similar relationship, but in a more
explicit way, and the problem could be solved elegantly.

3. Some people still can only use SVN. For that, GitHub has an SVN
interface [3] to the repositories.

4. We currently host our own SVN/Git, ViewVC and Klaus, Phabricator,
etc. Not only this incurs in additional admin cost, but it also gets
outdated, locally modified, and it needs to be backed up, etc. GitHub
gives all that for us for free.

5. We can still use Bugzilla (and lock GitHub's own bug system), but
we can also use GitHub's system to manage releases (it's actually
quite good for that).

6. GitHub has automated testing of merge requests, meaning we can have
pre-commit tests enabled on a set of fast bots, triggered by GitHub's
own validation hooks. Even though that wouldn't cover everything,
having a few pre-commit bots would considerably reduce the need to
revert patches [citation needed].

7. With git submodules, we'd probably want to follow the same style we
have today (llvm-projects/) instead of modelling how they look in
tree (llvm/tools/clang still as a symlink).

8. Once we're solo Git, we can shop around *much* more easily. By
using SVN, we're basically forced to host, or choose Source Forge.
Using just Git, we can choose GitLab, BitBucket and many others, if
GitHub is not appealing enough. Essentially, it doesn't matter where
you are, the tools are good, there and largely replaceable [citation
needed].

What do people think? Any issue not covered that we should? How would
that disrupt downstream users? Would it be a temporary disruption, but
with long lasting benefits? Or will it just break everything for you?

cheers,
--renato


[1]https://git-scm.com/book/en/v2/Git-Tools-Submodules
[2]https://help.github.com/articles/defining-the-mergeability-of-pull-requests/
[3]https://help.github.com/articles/support-for-subversion-clients/
___
cfe-dev mailing list
cfe-...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



--
John Criswell
Assistant Professor
Department of Computer Science, University of Rochester
http://www.cs.rochester.edu/u/criswell

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [cfe-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Tim Northover via lldb-dev
On 1 June 2016 at 10:12, Dan Liew via cfe-dev  wrote:
> the directories for each submodule will stay empty. Thus it isn't
> necessary to pull down all the sources when using git submodules.
> This would need support from the build system though. I'm not sure
> what the build system would do right now if it found an empty
> ``tools/clang`` directory in the source tree.

You can't have submodules within submodules like that (or at least it
didn't let me yesterday). For Unixy systems a simple "ln -s clang
llvm/tools" from the top-level should do the trick, but I vaguely
remember that Windows have issues with symlinks?

If so, we might have to restructure our CMake and we'd be able to fix
that at the same time.

Tim.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Renato Golin via lldb-dev
On 1 June 2016 at 18:12, Dan Liew  wrote:
>> So clone llvm ; cd tools / ; clone ...
>> --
>> good luck with this.. I foresee near religious opinions on the horizon..
>
> As do I.

Isn't that solved by the llvm-projects repository format?

--renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Daniel Berlin via lldb-dev
+1 from me

On Tue, May 31, 2016 at 12:51 PM, Tim Northover via llvm-dev <
llvm-...@lists.llvm.org> wrote:

> On 31 May 2016 at 12:31, Renato Golin via cfe-dev
>  wrote:
> > What do people think? Any issue not covered that we should?
>
> I'm in favour of the move. Git-svn just about works most of the time,
> but I find it makes committing to release branches particularly
> painful. It also randomly corrupts its database occasionally, just for
> the giggles I assume.
>
> Tim.
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [cfe-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Anton Korobeynikov via lldb-dev
>> Regarding the issue of git sub-modules and keeping Clang/LLVM in sync, 
>> perhaps we should just put Clang and LLVM into a single git repository and 
>> add a CMake option to disable compilation of Clang (the same could be done 
>> for other LLVM sub-projects for which bisection and other nifty features 
>> require a single revision number to refer to code across projects).  Keeping 
>> these projects in separate repositories is just more work, and I don't see 
>> what we're getting out of that extra effort.

I'm not sure we will benefit from having *all* the subprojects in the
single repository. This might affect pull / clone time significantly.
Think about having lld, lldb along with llvm + clang in the single
main repo.

-- 
With best regards, Anton Korobeynikov
Department of Statistical Modelling, Saint Petersburg State University
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Renato Golin via lldb-dev
On 1 June 2016 at 17:02, John Criswell  wrote:
> Do you have a set of volunteers lined up to do such a migration? Getting
> people willing to do the migration will obviously be key, and that was the
> one thing I didn't see in the original email.

Hi John,

Well, first we need to know if people are in favour, then if the
migration won't bring any serious problem, and then we can think of a
migration plan. :)

So far, it seems people are mostly in favour, with a few that reported
being locked into SVN. I had anticipated that, and have proposed
GitHub's SVN integration, which allows read-write access, so it should
be mostly ok. We need more tests on that side to be sure, though.

The biggest problem we're facing right now is how to sync the repos.
The existing llvm-repos format with all projects as sub-modules seem
to be a good candidate, but I still haven't seen a consensus on how
we'd do that.

About the migration plan, most people seem to agree a step-by-step
process is necessary. So, first we move to git-only, possibly with
sub-modules, then we move to GitHub/Lab/BB, then we move Phab to
GitHub merge requests, etc.

Regarding volunteers, I haven't yet asked around yet, but I'm sure we
have enough interested people to help. If everything else fails, I'm
more than happy to do all of that myself. Though, I'd have to learn a
lot more about sub-modules than I know today, which is effectively
nothing. :)

cheers,
--renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Mehdi Amini via lldb-dev

> On Jun 1, 2016, at 10:19 AM, Tim Northover via llvm-dev 
>  wrote:
> 
> On 1 June 2016 at 10:12, Dan Liew via cfe-dev  wrote:
>> the directories for each submodule will stay empty. Thus it isn't
>> necessary to pull down all the sources when using git submodules.
>> This would need support from the build system though. I'm not sure
>> what the build system would do right now if it found an empty
>> ``tools/clang`` directory in the source tree.
> 
> You can't have submodules within submodules like that (or at least it
> didn't let me yesterday). For Unixy systems a simple "ln -s clang
> llvm/tools" from the top-level should do the trick, but I vaguely
> remember that Windows have issues with symlinks?
> 
> If so, we might have to restructure our CMake and we'd be able to fix
> that at the same time.

CMake already supports not having clang in tools with 
-DLLVM_EXTERNAL_CLANG_SOURCE_DIR=path/to/clang

-- 
Mehdi

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Mehdi Amini via lldb-dev

> On Jun 1, 2016, at 11:18 AM, Renato Golin via llvm-dev 
>  wrote:
> 
> On 1 June 2016 at 17:02, John Criswell  wrote:
>> Do you have a set of volunteers lined up to do such a migration? Getting
>> people willing to do the migration will obviously be key, and that was the
>> one thing I didn't see in the original email.
> 
> Hi John,
> 
> Well, first we need to know if people are in favour, then if the
> migration won't bring any serious problem, and then we can think of a
> migration plan. :)
> 
> So far, it seems people are mostly in favour, with a few that reported
> being locked into SVN. I had anticipated that, and have proposed
> GitHub's SVN integration, which allows read-write access, so it should
> be mostly ok. We need more tests on that side to be sure, though.
> 
> The biggest problem we're facing right now is how to sync the repos.
> The existing llvm-repos format with all projects as sub-modules seem
> to be a good candidate, but I still haven't seen a consensus on how
> we'd do that.
> 
> About the migration plan, most people seem to agree a step-by-step
> process is necessary.

I personnally disagree with that, see below.

> So, first we move to git-only, possibly with
> sub-modules

If you move to git-only without the rest of the infrastructure/scripts, we're 
losing the convenience we have today with svn, and the "user experience" will 
not be so great. We may face resistance to this change.
I advocate to first set it up till it reaches the point of "good enough" in 
term of usability before actually migrating.

> , then we move to GitHub/Lab/BB,

It means we first need to host our git, write the tooling around it, to then 
migrate to github. I don't see the benefit over migrating directly to github: 
if people have to change their configuration, better have one single change.

> then we move Phab to
> GitHub merge requests, etc.

Phab to GitHub merge requests is a totally separate discussion IMO, and this 
discussion can happen after a successful move.


> 
> Regarding volunteers, I haven't yet asked around yet, but I'm sure we
> have enough interested people to help. If everything else fails, I'm
> more than happy to do all of that myself. Though, I'd have to learn a
> lot more about sub-modules than I know today, which is effectively
> nothing. :)

I'll be happy to throw manpower into tooling/infrastructure to make it happen.

-- 
Mehdi

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Aaron Ballman via lldb-dev
On Wed, Jun 1, 2016 at 2:18 PM, Renato Golin via llvm-dev
 wrote:
> On 1 June 2016 at 17:02, John Criswell  wrote:
>> Do you have a set of volunteers lined up to do such a migration? Getting
>> people willing to do the migration will obviously be key, and that was the
>> one thing I didn't see in the original email.
>
> Hi John,
>
> Well, first we need to know if people are in favour, then if the
> migration won't bring any serious problem, and then we can think of a
> migration plan. :)
>
> So far, it seems people are mostly in favour, with a few that reported
> being locked into SVN. I had anticipated that, and have proposed
> GitHub's SVN integration, which allows read-write access, so it should
> be mostly ok. We need more tests on that side to be sure, though.
>
> The biggest problem we're facing right now is how to sync the repos.
> The existing llvm-repos format with all projects as sub-modules seem
> to be a good candidate, but I still haven't seen a consensus on how
> we'd do that.
>
> About the migration plan, most people seem to agree a step-by-step
> process is necessary. So, first we move to git-only, possibly with
> sub-modules,

Despite people's reservations of a git-only repository? I mean, we
still don't know that this will even work for people who wish to stay
with SVN. I am really not comfortable with this decision based on "it
should be mostly ok" from above, but maybe I am misunderstanding
something.

~Aaron
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Matthias Braun via lldb-dev
So here's a straw-man proposal for a github migration:

1. Register an official github project with the llvm foundation.
2. Setup another (read-only) mirror of llvm.org/git at this github project
3. Make sure we have ala llvm-project-submodules setup in the official account. 
(Optional or necessary for the buildbots?) 
4. Update the buildbots to pick up updates and commits from the official git 
repository
5. Update phabricator to pick up commits from the official git repository
6. Tell people living downstream to pick up commits from the official git 
repository
7. Make sure bisecting with llvm-project-submodules is a good experience
8. Give things time to settle. We could play some games like disabling the svn 
repository for a few hours on purpose so that people can test that their 
infrastructure has really become independent of the svn repository.
9. Review and update llvm documentation.
10. Review website links pointing to viewvc/klaus/phab etc. to point to github 
instead.
... Until this point nothing has changed for developers, it will just boil down 
to a lot of work for buildbot and other infrstructure owners ...
11. Collect peoples github account information, give them push access. Ideally 
while still locking the github repository somehow...
12. Switch svn repository to read-only and allow pushs to the github repository.
13. Disable/remove/archive the svn repository.

- Matthias

> On Jun 1, 2016, at 11:31 AM, Mehdi Amini via llvm-dev 
>  wrote:
> 
> 
>> On Jun 1, 2016, at 11:18 AM, Renato Golin via llvm-dev 
>>  wrote:
>> 
>> On 1 June 2016 at 17:02, John Criswell  wrote:
>>> Do you have a set of volunteers lined up to do such a migration? Getting
>>> people willing to do the migration will obviously be key, and that was the
>>> one thing I didn't see in the original email.
>> 
>> Hi John,
>> 
>> Well, first we need to know if people are in favour, then if the
>> migration won't bring any serious problem, and then we can think of a
>> migration plan. :)
>> 
>> So far, it seems people are mostly in favour, with a few that reported
>> being locked into SVN. I had anticipated that, and have proposed
>> GitHub's SVN integration, which allows read-write access, so it should
>> be mostly ok. We need more tests on that side to be sure, though.
>> 
>> The biggest problem we're facing right now is how to sync the repos.
>> The existing llvm-repos format with all projects as sub-modules seem
>> to be a good candidate, but I still haven't seen a consensus on how
>> we'd do that.
>> 
>> About the migration plan, most people seem to agree a step-by-step
>> process is necessary.
> 
> I personnally disagree with that, see below.
> 
>> So, first we move to git-only, possibly with
>> sub-modules
> 
> If you move to git-only without the rest of the infrastructure/scripts, we're 
> losing the convenience we have today with svn, and the "user experience" will 
> not be so great. We may face resistance to this change.
> I advocate to first set it up till it reaches the point of "good enough" in 
> term of usability before actually migrating.
> 
>> , then we move to GitHub/Lab/BB,
> 
> It means we first need to host our git, write the tooling around it, to then 
> migrate to github. I don't see the benefit over migrating directly to github: 
> if people have to change their configuration, better have one single change.
> 
>> then we move Phab to
>> GitHub merge requests, etc.
> 
> Phab to GitHub merge requests is a totally separate discussion IMO, and this 
> discussion can happen after a successful move.
> 
> 
>> 
>> Regarding volunteers, I haven't yet asked around yet, but I'm sure we
>> have enough interested people to help. If everything else fails, I'm
>> more than happy to do all of that myself. Though, I'd have to learn a
>> lot more about sub-modules than I know today, which is effectively
>> nothing. :)
> 
> I'll be happy to throw manpower into tooling/infrastructure to make it happen.
> 
> -- 
> Mehdi
> 
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Mehdi Amini via lldb-dev

> On Jun 1, 2016, at 11:51 AM, Matthias Braun  wrote:
> 
> So here's a straw-man proposal for a github migration:
> 
> 1. Register an official github project with the llvm foundation.
> 2. Setup another (read-only) mirror of llvm.org/git at this github project
> 3. Make sure we have ala llvm-project-submodules setup in the official 
> account. (Optional or necessary for the buildbots?) 
> 4. Update the buildbots to pick up updates and commits from the official git 
> repository
> 5. Update phabricator to pick up commits from the official git repository
> 6. Tell people living downstream to pick up commits from the official git 
> repository
> 7. Make sure bisecting with llvm-project-submodules is a good experience
> 8. Give things time to settle. We could play some games like disabling the 
> svn repository for a few hours on purpose so that people can test that their 
> infrastructure has really become independent of the svn repository.
> 9. Review and update llvm documentation.
> 10. Review website links pointing to viewvc/klaus/phab etc. to point to 
> github instead.
> ... Until this point nothing has changed for developers, it will just boil 
> down to a lot of work for buildbot and other infrstructure owners ...
> 11. Collect peoples github account information, give them push access. 
> Ideally while still locking the github repository somehow...
> 12. Switch svn repository to read-only and allow pushs to the github 
> repository.

12.2: mirror git to svn :)

> 13. Disable/remove/archive the svn repository.


LGTM, this is how I'd approach it as well, thanks for writing it down!

-- 
Mehdi

> 
> - Matthias
> 
>> On Jun 1, 2016, at 11:31 AM, Mehdi Amini via llvm-dev 
>>  wrote:
>> 
>> 
>>> On Jun 1, 2016, at 11:18 AM, Renato Golin via llvm-dev 
>>>  wrote:
>>> 
>>> On 1 June 2016 at 17:02, John Criswell  wrote:
 Do you have a set of volunteers lined up to do such a migration? Getting
 people willing to do the migration will obviously be key, and that was the
 one thing I didn't see in the original email.
>>> 
>>> Hi John,
>>> 
>>> Well, first we need to know if people are in favour, then if the
>>> migration won't bring any serious problem, and then we can think of a
>>> migration plan. :)
>>> 
>>> So far, it seems people are mostly in favour, with a few that reported
>>> being locked into SVN. I had anticipated that, and have proposed
>>> GitHub's SVN integration, which allows read-write access, so it should
>>> be mostly ok. We need more tests on that side to be sure, though.
>>> 
>>> The biggest problem we're facing right now is how to sync the repos.
>>> The existing llvm-repos format with all projects as sub-modules seem
>>> to be a good candidate, but I still haven't seen a consensus on how
>>> we'd do that.
>>> 
>>> About the migration plan, most people seem to agree a step-by-step
>>> process is necessary.
>> 
>> I personnally disagree with that, see below.
>> 
>>> So, first we move to git-only, possibly with
>>> sub-modules
>> 
>> If you move to git-only without the rest of the infrastructure/scripts, 
>> we're losing the convenience we have today with svn, and the "user 
>> experience" will not be so great. We may face resistance to this change.
>> I advocate to first set it up till it reaches the point of "good enough" in 
>> term of usability before actually migrating.
>> 
>>> , then we move to GitHub/Lab/BB,
>> 
>> It means we first need to host our git, write the tooling around it, to then 
>> migrate to github. I don't see the benefit over migrating directly to 
>> github: if people have to change their configuration, better have one single 
>> change.
>> 
>>> then we move Phab to
>>> GitHub merge requests, etc.
>> 
>> Phab to GitHub merge requests is a totally separate discussion IMO, and this 
>> discussion can happen after a successful move.
>> 
>> 
>>> 
>>> Regarding volunteers, I haven't yet asked around yet, but I'm sure we
>>> have enough interested people to help. If everything else fails, I'm
>>> more than happy to do all of that myself. Though, I'd have to learn a
>>> lot more about sub-modules than I know today, which is effectively
>>> nothing. :)
>> 
>> I'll be happy to throw manpower into tooling/infrastructure to make it 
>> happen.
>> 
>> -- 
>> Mehdi
>> 
>> ___
>> LLVM Developers mailing list
>> llvm-...@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Renato Golin via lldb-dev
On 1 June 2016 at 19:31, Mehdi Amini  wrote:
> If you move to git-only without the rest of the infrastructure/scripts, we're 
> losing the convenience we have today with svn, and the "user experience" will 
> not be so great. We may face resistance to this change.
> I advocate to first set it up till it reaches the point of "good enough" in 
> term of usability before actually migrating.

That's what I meant... :)


> It means we first need to host our git, write the tooling around it, to then 
> migrate to github. I don't see the benefit over migrating directly to github: 
> if people have to change their configuration, better have one single change.

We already host our own git, so people already use it. Moving the git
repo location will also disrupt infrastructure, and is not critical,
so we can do it later.


> Phab to GitHub merge requests is a totally separate discussion IMO, and this 
> discussion can happen after a successful move.

Indeed, what I meant as well. :)


> I'll be happy to throw manpower into tooling/infrastructure to make it happen.

Thanks! Most appreciated!

--renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Renato Golin via lldb-dev
On 1 June 2016 at 19:36, Aaron Ballman  wrote:
> Despite people's reservations of a git-only repository?

Hi Aaron, not at all!

I was especially vague on my first email to make sure SVN folks would
be shoved on the side, but John had asked for a full plan *in the case
we move*, and I was just completing the picture.

Having said that, I can't take that decision alone, and my own opinion
is irrelevant on the grand scheme.

Right now, our main repo is in SVN with most people using Git. If the
vast majority vote for the move, it wouldn't be fair to continue to
force SVN on them, and it would be overall less effort for the few
people that prefer SVN to have a bit more work than they have today,
to save the majority of Git users the extra work. I have no idea how
much people is enough to move to Git, but unless we fix the sub-module
problem, there's no point in even trying.

So, my personal points are:

1. We can only move IFF the Git solution is technically equivalent or
superior than what we have today.

2. We should only move IFF the vast majority will see benefits from
it, even if a small minority will see some increased effort. Of
course, the balance of efforts has to be overall positive.

3. We should not move if there is no replacement for SVN users at the
moment. We should try to encourage SVN users to move to Git, to speed
up the move, though.

I'm assuming the SVN vs. Git argument is not just a personal thing,
but a tooling / infrastructure issue. The bigger picture here is not
which VCS is better, but getting rid of a huge infrastructure cost
from our part, which nowadays means moving to Git or using
SourceForge.

cheers,
--renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Renato Golin via lldb-dev
On 1 June 2016 at 19:55, Mehdi Amini  wrote:
> 12.2: mirror git to svn :)

either that or use GitHub's SVN interface, which is RW.

--renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Mehdi Amini via lldb-dev

> On Jun 1, 2016, at 11:36 AM, Aaron Ballman via llvm-dev 
>  wrote:
> 
> On Wed, Jun 1, 2016 at 2:18 PM, Renato Golin via llvm-dev
>  wrote:
>> On 1 June 2016 at 17:02, John Criswell  wrote:
>>> Do you have a set of volunteers lined up to do such a migration? Getting
>>> people willing to do the migration will obviously be key, and that was the
>>> one thing I didn't see in the original email.
>> 
>> Hi John,
>> 
>> Well, first we need to know if people are in favour, then if the
>> migration won't bring any serious problem, and then we can think of a
>> migration plan. :)
>> 
>> So far, it seems people are mostly in favour, with a few that reported
>> being locked into SVN. I had anticipated that, and have proposed
>> GitHub's SVN integration, which allows read-write access, so it should
>> be mostly ok. We need more tests on that side to be sure, though.
>> 
>> The biggest problem we're facing right now is how to sync the repos.
>> The existing llvm-repos format with all projects as sub-modules seem
>> to be a good candidate, but I still haven't seen a consensus on how
>> we'd do that.
>> 
>> About the migration plan, most people seem to agree a step-by-step
>> process is necessary. So, first we move to git-only, possibly with
>> sub-modules,
> 
> Despite people's reservations of a git-only repository? I mean, we
> still don't know that this will even work for people who wish to stay
> with SVN.

The thread is open on llvm-dev, there won't be any migration next week (or even 
next month), that leaves time for any contributor to raise any real blockers.
I haven't seen any of these till now, but the thread is getting long and  I may 
have missed an email..

-- 
Mehdi


> I am really not comfortable with this decision based on "it
> should be mostly ok" from above, but maybe I am misunderstanding
> something.
> 
> ~Aaron
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Richard Smith via lldb-dev
On Wed, Jun 1, 2016 at 11:18 AM, Renato Golin via llvm-dev <
llvm-...@lists.llvm.org> wrote:

> On 1 June 2016 at 17:02, John Criswell  wrote:
> > Do you have a set of volunteers lined up to do such a migration? Getting
> > people willing to do the migration will obviously be key, and that was
> the
> > one thing I didn't see in the original email.
>
> Hi John,
>
> Well, first we need to know if people are in favour, then if the
> migration won't bring any serious problem, and then we can think of a
> migration plan. :)
>
> So far, it seems people are mostly in favour, with a few that reported
> being locked into SVN. I had anticipated that, and have proposed
> GitHub's SVN integration, which allows read-write access, so it should
> be mostly ok. We need more tests on that side to be sure, though.
>
> The biggest problem we're facing right now is how to sync the repos.
> The existing llvm-repos format with all projects as sub-modules seem
> to be a good candidate, but I still haven't seen a consensus on how
> we'd do that.
>

Is it possible to set up a post-commit hook for each child repository that
commits the new child repo's head into the corresponding branch of the
parent repository? Does anyone have experience with doing that on github?
That would seem to give the workflow that we want.

About the migration plan, most people seem to agree a step-by-step
> process is necessary. So, first we move to git-only, possibly with
> sub-modules, then we move to GitHub/Lab/BB, then we move Phab to
> GitHub merge requests, etc.
>
> Regarding volunteers, I haven't yet asked around yet, but I'm sure we
> have enough interested people to help. If everything else fails, I'm
> more than happy to do all of that myself. Though, I'd have to learn a
> lot more about sub-modules than I know today, which is effectively
> nothing. :)
>
> cheers,
> --renato
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Aaron Ballman via lldb-dev
On Wed, Jun 1, 2016 at 3:10 PM, Renato Golin  wrote:
> On 1 June 2016 at 19:36, Aaron Ballman  wrote:
>> Despite people's reservations of a git-only repository?
>
> Hi Aaron, not at all!
>
> I was especially vague on my first email to make sure SVN folks would
> be shoved on the side, but John had asked for a full plan *in the case
> we move*, and I was just completing the picture.

Oh! That makes perfect sense to me. Thank you for the clarification!

> Having said that, I can't take that decision alone, and my own opinion
> is irrelevant on the grand scheme.
>
> Right now, our main repo is in SVN with most people using Git.

Our main repo is in SVN; I would say we don't know what most people
are using (aside from "svn for write access because it's the only
option").

> If the
> vast majority vote for the move, it wouldn't be fair to continue to
> force SVN on them, and it would be overall less effort for the few
> people that prefer SVN to have a bit more work than they have today,
> to save the majority of Git users the extra work. I have no idea how
> much people is enough to move to Git, but unless we fix the sub-module
> problem, there's no point in even trying.

Fair points, but with the caveat that people using git today have a
workable solution (as I understand it, and I could be totally wrong)
using the git mirrors. That's not a reason to not transition from svn
to git, however.

> So, my personal points are:
>
> 1. We can only move IFF the Git solution is technically equivalent or
> superior than what we have today.
>
> 2. We should only move IFF the vast majority will see benefits from
> it, even if a small minority will see some increased effort. Of
> course, the balance of efforts has to be overall positive.

Agreed with both of these points.

> 3. We should not move if there is no replacement for SVN users at the
> moment.

Agreed.

> We should try to encourage SVN users to move to Git, to speed
> up the move, though.

This is implying that we will move, which I think should still be left
as a vague question mark until we have more answers. Based on that, I
think it's premature to encourage anyone to switch to git.

> I'm assuming the SVN vs. Git argument is not just a personal thing,
> but a tooling / infrastructure issue. The bigger picture here is not
> which VCS is better, but getting rid of a huge infrastructure cost
> from our part, which nowadays means moving to Git or using
> SourceForge.

I agree that there are infrastructure costs to consider. I just hope
we don't consider those at the expense of a functioning system that
people are used to using and already have workflows based on. Git may
be the new shiny, but it's not what we use today for LLVM. It's
sometimes easy to forget there's also cost with telling the community
"please go learn a new, very different toolset so that you can
continue to contribute to the project." I'm not making a claim that
the costs aren't worth the gains (because this might very well be the
correct time to switch VCS), but I am worried when emails make it
sound like switching to git-only is a foregone conclusion, which is a
bit of a strange way to start a discussion about whether the community
wants to switch. That being said, I like that we're discussing what a
switch would look like were one to occur so that we can suss out all
the pros and cons!

~Aaron
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Renato Golin via lldb-dev
On 1 June 2016 at 20:31, Aaron Ballman  wrote:
> Our main repo is in SVN; I would say we don't know what most people
> are using (aside from "svn for write access because it's the only
> option").

If the LLVM Meetings are any indication, and they are at least related
to the most active developers, everyone I've asked (and I did ask a
lot over the last 2 years) was using Git for development. Most people
were using Git-SVN for commits, with some few still using a separate
SVN repository.

I'm not really too worried about personal preferences. The
infrastructure cost alone is more important than any of our
preferences. That is why I have been using SVN all these years, even
though I hate it more than I once hated CVS.

The admin cost is high, and we're not sysadmins, so GitHub would
provide an *immense* value for the price of $0. I can't argue we that.

But there are also downstream infrastructure issues that need to be
taken into account. As James said, knowing Git well is almost a
required skill nowadays, and everything is done thinking about Git
these days, so the likelihood that you'll find a replacement for your
company's infrastructure to work with Git is higher than a new process
will work well with SVN. In the long term, SVN will be just like CVS
was 10 years ago or RCS 20. It'll be impossible to work with them, and
companies depending on that infrastructure will be in *serious*
trouble.



> Fair points, but with the caveat that people using git today have a
> workable solution (as I understand it, and I could be totally wrong)
> using the git mirrors. That's not a reason to not transition from svn
> to git, however.

Indeed. It's all about the overall costs for the whole community.
Personal preferences will be mostly diluted, with strong technical
arguments being the driving forces.


> This is implying that we will move, which I think should still be left
> as a vague question mark until we have more answers. Based on that, I
> think it's premature to encourage anyone to switch to git.

I will (only slightly and personally) disagree, based on James' point
of how important Git is today.

Ever since I discovered Git I have always encouraged people to use it,
regardless, and I'll keep encouraging. :)

But that's irrelevant to the discussion.


> but I am worried when emails make it
> sound like switching to git-only is a foregone conclusion, which is a
> bit of a strange way to start a discussion about whether the community
> wants to switch.

I think that's just the result of people enthusiastic with the
opportunity to move to a better working environment. Not everyone is,
but those that are, are showing.

This is another indication that there are more Git users than SVN
users, but not an argument to force the move.

The only arguments we should accept are technical ones.

cheers,
--renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Richard Smith via lldb-dev
On Wed, Jun 1, 2016 at 1:07 PM, Manuel Jacob via llvm-dev <
llvm-...@lists.llvm.org> wrote:

> On 2016-05-31 22:45, Mehdi Amini via llvm-dev wrote:
>
>> On May 31, 2016, at 1:31 PM, Renato Golin 
>>> wrote:
>>>
>>> On 31 May 2016 at 21:28, Mehdi Amini  wrote:
>>>
 Ideally, I'd prefer the cross-repository to be handled with an extra
 layer, in a way similar as described in:
 https://gerrit-review.googlesource.com/Documentation/user-submodules.htm
 (somehow conceptually similar to Android manifests XML files).
 It would be easy to have tooling/scripts for llvm that would easily say
 "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or "update all
 llvm subproject under this root", or "checkout this specific revision for
 all these" (with a monotonic number for the revision).

>>>
>>> At Linaro, we already have a set of scripts that do that. We're now
>>> moving to git worktree, and I think it's going to simplify our work
>>> considerably. But honestly, I'd rather not force anyone to use any set
>>> of scripts, and let people work directly with git, so I'd be more in
>>> favour of having a server-side solution, if at all possible.
>>>
>>
>> Apparently I wasn't very clear: llvm and clang (and the others
>> projects) would be simple decoupled, individual git repositories. You
>> would be able to check them out however you want and commit to them
>> individually.
>> There would be an extra "integration repository" on top that would
>> only provide the service that tells "r12345 is llvm:36c941c
>> clang:eaf492b compiler-rt:6d77ea5". This repository should be managed
>> transparently by some server-side integration.
>> The provided scripting I was referring to would just be a convenience
>> that is using this extra layer of metadata ("integration repository")
>> to be able checkout the other individual repositories together at the
>> right "rev-lock" revision.
>> This is not on your way if you don't want to use it, but it provides
>> this "single increase monotonic revision number across multiple
>> repository" that is convenient for some people.
>>
>> Makes sense?
>>
>
> How would you ensure that two dependent changes on different repositories
> get the same revision number?


That is not the case today and isn't (in my opinion) a problem we need to
solve with this migration. If there's a small (1-2 revision) window in
which things are broken, that's annoying but not a showstopper and not a
regression.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Mehdi Amini via lldb-dev


> On Jun 1, 2016, at 1:07 PM, Manuel Jacob  wrote:
> 
> On 2016-05-31 22:45, Mehdi Amini via llvm-dev wrote:
>>> On May 31, 2016, at 1:31 PM, Renato Golin  wrote:
 On 31 May 2016 at 21:28, Mehdi Amini  wrote:
 Ideally, I'd prefer the cross-repository to be handled with an extra 
 layer, in a way similar as described in: 
 https://gerrit-review.googlesource.com/Documentation/user-submodules.htm 
 (somehow conceptually similar to Android manifests XML files).
 It would be easy to have tooling/scripts for llvm that would easily say 
 "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or "update all 
 llvm subproject under this root", or "checkout this specific revision for 
 all these" (with a monotonic number for the revision).
>>> At Linaro, we already have a set of scripts that do that. We're now
>>> moving to git worktree, and I think it's going to simplify our work
>>> considerably. But honestly, I'd rather not force anyone to use any set
>>> of scripts, and let people work directly with git, so I'd be more in
>>> favour of having a server-side solution, if at all possible.
>> Apparently I wasn't very clear: llvm and clang (and the others
>> projects) would be simple decoupled, individual git repositories. You
>> would be able to check them out however you want and commit to them
>> individually.
>> There would be an extra "integration repository" on top that would
>> only provide the service that tells "r12345 is llvm:36c941c
>> clang:eaf492b compiler-rt:6d77ea5". This repository should be managed
>> transparently by some server-side integration.
>> The provided scripting I was referring to would just be a convenience
>> that is using this extra layer of metadata ("integration repository")
>> to be able checkout the other individual repositories together at the
>> right "rev-lock" revision.
>> This is not on your way if you don't want to use it, but it provides
>> this "single increase monotonic revision number across multiple
>> repository" that is convenient for some people.
>> Makes sense?
> 
> How would you ensure that two dependent changes on different repositories get 
> the same revision number?

I don't plan to try to address that. I plan to reproduce the behavior we have 
right now when someone commit an API change in LLVM and separately commit the 
fix in clang: the fix gets a different revision number.
I'm not sure how to be robust against that other than putting all the projects 
in the same repo and asking developers to build them all before push.

If you have ideas/suggestions, please share!

-- 
Mehdi
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Renato Golin via lldb-dev
On 1 June 2016 at 21:21, Mehdi Amini  wrote:
> I'm not sure how to be robust against that other than putting all the 
> projects in the same repo and asking developers to build them all before push.

I'm strongly against a single repo with all in, or asking to build
LLDB when the commit is in Compiler-RT. As usual, appropriate tests
are all we need.

I share your view that we already have a window of possible breakage
due to co-dependant commits and therefore can afford the same
behaviour in Git.

IIGIR, sub-modules are meant to coordinate the repositories, not to
solve the problematic short window of doom. Even having a single
repository with all in won't help this problem.

Unless there is a way to link two commits to two different submodules
and make the bots know that they can't be tested separately, we should
just accept that some bots will break if they're extremely unlucky.

Having said that, pre-commit bots will break with most co-dependant
patches, and we must have a way to either ignore those failures and
still merge, or to mark them as co-dependants and test again.

cheers,
--renato
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [cfe-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread James Y Knight via lldb-dev
IMO, if we're switching to git, we should just be clear up front that all
committers will be expected to switch to git as well -- or at least, if
they want to use something else (e.g. mercurial's git bridge/etc), that
it's their own problem.

It is truly NOT that big an imposition to require the use of git. And
knowing how to use git at at least a basic level is an important skill for
a lot of software development now -- no matter what LLVM does, so I don't
feel bad for making anyone spend time learning how to use it.

I really don't think that promising and requiring that svn-client using
people (especially committers: read-only access seems a lot less
potentially problematic) will keep getting a good development experience
after the migration is a good idea. I mean, if SVN also happens to work
with the chosen hosting/workflow in the end, that's fine, I guess. But, I
feel that should be considered a "if it works, that's okay, but it's not
recommended, and is not guaranteed" kind of thing.

Making that a requirement locks us into the use of github as the primary
repository: no other git hosting has svn support, afaik.
It means we can't introduce any workflows that wouldn't work well for svn
users -- or if we do, that such users will probably complain anew when that
happens.
And if github's svn bridge turns out to have fatal problems, do we then
abandon the migration?



On Wed, Jun 1, 2016 at 2:36 PM, Aaron Ballman via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> On Wed, Jun 1, 2016 at 2:18 PM, Renato Golin via llvm-dev
>  wrote:
> > On 1 June 2016 at 17:02, John Criswell  wrote:
> >> Do you have a set of volunteers lined up to do such a migration? Getting
> >> people willing to do the migration will obviously be key, and that was
> the
> >> one thing I didn't see in the original email.
> >
> > Hi John,
> >
> > Well, first we need to know if people are in favour, then if the
> > migration won't bring any serious problem, and then we can think of a
> > migration plan. :)
> >
> > So far, it seems people are mostly in favour, with a few that reported
> > being locked into SVN. I had anticipated that, and have proposed
> > GitHub's SVN integration, which allows read-write access, so it should
> > be mostly ok. We need more tests on that side to be sure, though.
> >
> > The biggest problem we're facing right now is how to sync the repos.
> > The existing llvm-repos format with all projects as sub-modules seem
> > to be a good candidate, but I still haven't seen a consensus on how
> > we'd do that.
> >
> > About the migration plan, most people seem to agree a step-by-step
> > process is necessary. So, first we move to git-only, possibly with
> > sub-modules,
>
> Despite people's reservations of a git-only repository? I mean, we
> still don't know that this will even work for people who wish to stay
> with SVN. I am really not comfortable with this decision based on "it
> should be mostly ok" from above, but maybe I am misunderstanding
> something.
>
> ~Aaron
> ___
> cfe-dev mailing list
> cfe-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Manuel Jacob via lldb-dev

On 2016-05-31 22:45, Mehdi Amini via llvm-dev wrote:
On May 31, 2016, at 1:31 PM, Renato Golin  
wrote:


On 31 May 2016 at 21:28, Mehdi Amini  wrote:
Ideally, I'd prefer the cross-repository to be handled with an extra 
layer, in a way similar as described in: 
https://gerrit-review.googlesource.com/Documentation/user-submodules.htm 
(somehow conceptually similar to Android manifests XML files).
It would be easy to have tooling/scripts for llvm that would easily 
say "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or 
"update all llvm subproject under this root", or "checkout this 
specific revision for all these" (with a monotonic number for the 
revision).


At Linaro, we already have a set of scripts that do that. We're now
moving to git worktree, and I think it's going to simplify our work
considerably. But honestly, I'd rather not force anyone to use any set
of scripts, and let people work directly with git, so I'd be more in
favour of having a server-side solution, if at all possible.


Apparently I wasn't very clear: llvm and clang (and the others
projects) would be simple decoupled, individual git repositories. You
would be able to check them out however you want and commit to them
individually.
There would be an extra "integration repository" on top that would
only provide the service that tells "r12345 is llvm:36c941c
clang:eaf492b compiler-rt:6d77ea5". This repository should be managed
transparently by some server-side integration.
The provided scripting I was referring to would just be a convenience
that is using this extra layer of metadata ("integration repository")
to be able checkout the other individual repositories together at the
right "rev-lock" revision.
This is not on your way if you don't want to use it, but it provides
this "single increase monotonic revision number across multiple
repository" that is convenient for some people.

Makes sense?


How would you ensure that two dependent changes on different 
repositories get the same revision number?

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Bruce Hoult via lldb-dev
With submodules, the current hash of each submodule is recorded in each
master commit. If you check out a different master repository commit then
you run 'git submodule update' and it checks out the corresponding commit
in each submodule. I'm not sure why this isn't automatic with the master
repo commit checkout, but in any case it's not difficult.

So when you do a git bisect, you just need to make sure there is a 'git
submodule update' at the start of your bisect script.

It will be fun to get all the correct hashes recorded in the master repo
during initial import. But not all that difficult: check out next svn
revision; check and push in any submodules with changes in that revision;
then commit the master repo for that revision.


On Wed, Jun 1, 2016 at 2:06 AM, Robinson, Paul via llvm-dev <
llvm-...@lists.llvm.org> wrote:

>
>
> > -Original Message-
> > From: mehdi.am...@apple.com [mailto:mehdi.am...@apple.com]
> > Sent: Tuesday, May 31, 2016 3:54 PM
> > To: Robinson, Paul
> > Cc: Bill Kelly; Clang Dev; LLDB Dev; llvm-...@lists.llvm.org
> > Subject: Re: [llvm-dev] GitHub anyone?
> >
> >
> > > On May 31, 2016, at 3:38 PM, Robinson, Paul 
> > wrote:
> > >
> > >
> > >
> > >> -Original Message-
> > >> From: llvm-dev [mailto:llvm-dev-boun...@lists.llvm.org] On Behalf Of
> > Mehdi
> > >> Amini via llvm-dev
> > >> Sent: Tuesday, May 31, 2016 2:38 PM
> > >> To: Bill Kelly
> > >> Cc: LLVM Dev; Clang Dev; LLDB Dev
> > >> Subject: Re: [llvm-dev] GitHub anyone?
> > >>
> > >>
> > >>> On May 31, 2016, at 2:01 PM, Bill Kelly via llvm-dev  > >> d...@lists.llvm.org> wrote:
> > >>>
> > >>> Chris Lattner via llvm-dev wrote:
> >  Personally, I’m hugely in favor of moving llvm’s source hosting to
> > >> github at
> >  some point, despite the fact that I continue to dislike git as a
> tool
> > >> and
> >  consider monotonicly increasing version numbers to be hugely
> > >> beneficial.
> > >>>
> > >>> For whatever it's worth, our projects define a `buildnum` git alias:
> > >>>
> > >>> alias.buildnum=!sh -c "git rev-list --all | wc -l"
> > >
> > > Or the cheaper "git rev-list --count --all" if your git is new enough.
> > > We do something like this as well.
> > >
> > >>>
> > >>> So from the shell:
> > >>>
> > >>> $ git buildnum
> > >>> 17475
> > >>>
> > >>> This number increases monotonically per commit.
> > >>
> > >> It does not work with branches though (we're not really planning to
> > have
> > >> branches I believe),
> > >
> > > You can get a per-branch unique number with this tactic.  On our local
> > > branches we use "rev-list origin/master.." which is the number of
> > commits
> > > since branching from master, and that's enough for our local purposes.
> > >
> > >> but more importantly it won't handle cross-repository
> > >> versioning (how do you relate the number this command prints in the
> > llvm
> > >> repo to the number it'll print in the clang repo?), which I believe is
> > >> something important considering our setup.
> > >
> > > Is it really that important?  Or are we just used to the convenience?
> >
> > I don't know how important it is. How would you bisect without this
> > "convenience" for instance?
> > (There is nothing like "push date" in git)
>
> I know that on a single branch, "git bisect" deals with that for you.
> I've seen the talk about submodules but I have no clue how that works
> or whether git-bisect can operate cleanly in that situation.
> --paulr
>
> >
> > --
> > Mehdi
> >
> >
> > > If the Clang build number is a tuple (cfe-number, llvm-number) instead
> > > of a single number, how horrible is that really?  If you consider what
> > > an out-of-tree front end probably does, it's exactly the same thing.
> > >
> > > (I admit that locally we mush cfe+llvm into a single branch and do the
> > > rev-list count to get a single number. But that's more for our own
> > > convenience than anything else.)
> > > --paulr
> > >
> > >>
> > >> --
> > >> Mehdi
> > >>
> > >>
> > >>
> > >>>
> > >>>
> > >>> Our build scripts make this number available in various #define
> forms.
> > >>>
> > >>> (We use a little extra scripting logic to also determine whether
> there
> > >>> are currently any unmerged or uncommitted changes, and add an
> > annotation
> > >>> to the program version in that case, e.g.  "9.3.17475 [unmerged]")
> > >>>
> > >>>
> > >>> It's all stupidly simple, but seems to work well enough for us.
> > >>>
> > >>>
> > >>>
> > >>> Regards,
> > >>>
> > >>> Bill
> > >>>
> > >>> ___
> > >>> LLVM Developers mailing list
> > >>> llvm-...@lists.llvm.org
> > >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > >>
> > >> ___
> > >> LLVM Developers mailing list
> > >> llvm-...@lists.llvm.org
> > >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi

Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Brian M. Rzycki via lldb-dev
+1 from me. Our company has very restrictive firewalls and the proxy setup
doesn't work with SVN and http. We have to use special machines in a DMZ
when we push changes to SVN.

To mitigate this we are able to use the individual git repos but that poses
problems for regression testing when we attempt to use git bisect. We
either have to try and find a "next nearest" commit for clang, compiler-rt,
etc or use the unified repository llvm-project on github already.

Lastly, it's non-trivial from only the Git repos to reconstruct the code
that goes into a released compiler. I'd hope with this new submodule layout
it'd be as simple as git branch llvm-3.10-release and cmake.


On Wed, Jun 1, 2016 at 1:51 PM, Matthias Braun via llvm-dev <
llvm-...@lists.llvm.org> wrote:

> So here's a straw-man proposal for a github migration:
>
> 1. Register an official github project with the llvm foundation.
> 2. Setup another (read-only) mirror of llvm.org/git at this github project
> 3. Make sure we have ala llvm-project-submodules setup in the official
> account. (Optional or necessary for the buildbots?)
> 4. Update the buildbots to pick up updates and commits from the official
> git repository
> 5. Update phabricator to pick up commits from the official git repository
> 6. Tell people living downstream to pick up commits from the official git
> repository
> 7. Make sure bisecting with llvm-project-submodules is a good experience
> 8. Give things time to settle. We could play some games like disabling the
> svn repository for a few hours on purpose so that people can test that
> their infrastructure has really become independent of the svn repository.
> 9. Review and update llvm documentation.
> 10. Review website links pointing to viewvc/klaus/phab etc. to point to
> github instead.
> ... Until this point nothing has changed for developers, it will just boil
> down to a lot of work for buildbot and other infrstructure owners ...
> 11. Collect peoples github account information, give them push access.
> Ideally while still locking the github repository somehow...
> 12. Switch svn repository to read-only and allow pushs to the github
> repository.
> 13. Disable/remove/archive the svn repository.
>
> - Matthias
>
> > On Jun 1, 2016, at 11:31 AM, Mehdi Amini via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
> >
> >
> >> On Jun 1, 2016, at 11:18 AM, Renato Golin via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
> >>
> >> On 1 June 2016 at 17:02, John Criswell  wrote:
> >>> Do you have a set of volunteers lined up to do such a migration?
> Getting
> >>> people willing to do the migration will obviously be key, and that was
> the
> >>> one thing I didn't see in the original email.
> >>
> >> Hi John,
> >>
> >> Well, first we need to know if people are in favour, then if the
> >> migration won't bring any serious problem, and then we can think of a
> >> migration plan. :)
> >>
> >> So far, it seems people are mostly in favour, with a few that reported
> >> being locked into SVN. I had anticipated that, and have proposed
> >> GitHub's SVN integration, which allows read-write access, so it should
> >> be mostly ok. We need more tests on that side to be sure, though.
> >>
> >> The biggest problem we're facing right now is how to sync the repos.
> >> The existing llvm-repos format with all projects as sub-modules seem
> >> to be a good candidate, but I still haven't seen a consensus on how
> >> we'd do that.
> >>
> >> About the migration plan, most people seem to agree a step-by-step
> >> process is necessary.
> >
> > I personnally disagree with that, see below.
> >
> >> So, first we move to git-only, possibly with
> >> sub-modules
> >
> > If you move to git-only without the rest of the infrastructure/scripts,
> we're losing the convenience we have today with svn, and the "user
> experience" will not be so great. We may face resistance to this change.
> > I advocate to first set it up till it reaches the point of "good enough"
> in term of usability before actually migrating.
> >
> >> , then we move to GitHub/Lab/BB,
> >
> > It means we first need to host our git, write the tooling around it, to
> then migrate to github. I don't see the benefit over migrating directly to
> github: if people have to change their configuration, better have one
> single change.
> >
> >> then we move Phab to
> >> GitHub merge requests, etc.
> >
> > Phab to GitHub merge requests is a totally separate discussion IMO, and
> this discussion can happen after a successful move.
> >
> >
> >>
> >> Regarding volunteers, I haven't yet asked around yet, but I'm sure we
> >> have enough interested people to help. If everything else fails, I'm
> >> more than happy to do all of that myself. Though, I'd have to learn a
> >> lot more about sub-modules than I know today, which is effectively
> >> nothing. :)
> >
> > I'll be happy to throw manpower into tooling/infrastructure to make it
> happen.
> >
> > --
> > Mehdi
> >
> > ___
> >

Re: [lldb-dev] [cfe-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Dan Liew via lldb-dev
On 1 June 2016 at 10:19, Tim Northover  wrote:
> On 1 June 2016 at 10:12, Dan Liew via cfe-dev  wrote:
>> the directories for each submodule will stay empty. Thus it isn't
>> necessary to pull down all the sources when using git submodules.
>> This would need support from the build system though. I'm not sure
>> what the build system would do right now if it found an empty
>> ``tools/clang`` directory in the source tree.
>
> You can't have submodules within submodules like that (or at least it
> didn't let me yesterday).

I didn't say anything about nested submodules. I was imagining the
LLVM repo being the top level repository with submodules for

projects/libcxx
projects/libcxxabi
projects/compiler-rt
tools/clang
tools/lldb

etc..

There's no nesting of submodules here. This assumes an in source tree
nesting of projects which perhaps is something we shouldn't do given
that the projects can also be built outside the source tree.

However I should say given previous experience with submodules I am a
little bit wary using them because of the problems caused by
developers not using them properly and breaking stuff.

I like the sounds of Mehdi's proposal of an automated integration
repository although I would need to look at the details a little
closer.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Jim Rowan via lldb-dev
+1

We use git exclusively within QC, so this looks like simplification to us.   
There was mention early in the thread of continuing to enforce linear history; 
that’s important to our internal integration machinery.   We do currently use 
the git-svn-id as a key for some of our internal processes, but it won’t be a 
problem to switch to something else.   

We use google repo to stitch together our build tree, so the submodule 
discussion is mostly a no-op for us (providing it ends up being the 
“llvm-project” flat tree implementation).

On Jun 1, 2016, at 1:51 PM, Matthias Braun via llvm-dev 
 wrote:

> So here's a straw-man proposal for a github migration:
> 
> 1. Register an official github project with the llvm foundation.
> 2. Setup another (read-only) mirror of llvm.org/git at this github project
> 3. Make sure we have ala llvm-project-submodules setup in the official 
> account. (Optional or necessary for the buildbots?) 
> 4. Update the buildbots to pick up updates and commits from the official git 
> repository
> 5. Update phabricator to pick up commits from the official git repository
> 6. Tell people living downstream to pick up commits from the official git 
> repository
> 7. Make sure bisecting with llvm-project-submodules is a good experience
> 8. Give things time to settle. We could play some games like disabling the 
> svn repository for a few hours on purpose so that people can test that their 
> infrastructure has really become independent of the svn repository.
> 9. Review and update llvm documentation.
> 10. Review website links pointing to viewvc/klaus/phab etc. to point to 
> github instead.
> ... Until this point nothing has changed for developers, it will just boil 
> down to a lot of work for buildbot and other infrstructure owners ...
> 11. Collect peoples github account information, give them push access. 
> Ideally while still locking the github repository somehow...
> 12. Switch svn repository to read-only and allow pushs to the github 
> repository.
> 13. Disable/remove/archive the svn repository.
> 
> - Matthias
> 
>> On Jun 1, 2016, at 11:31 AM, Mehdi Amini via llvm-dev 
>>  wrote:
>> 
>> 
>>> On Jun 1, 2016, at 11:18 AM, Renato Golin via llvm-dev 
>>>  wrote:
>>> 
>>> On 1 June 2016 at 17:02, John Criswell  wrote:
 Do you have a set of volunteers lined up to do such a migration? Getting
 people willing to do the migration will obviously be key, and that was the
 one thing I didn't see in the original email.
>>> 
>>> Hi John,
>>> 
>>> Well, first we need to know if people are in favour, then if the
>>> migration won't bring any serious problem, and then we can think of a
>>> migration plan. :)
>>> 
>>> So far, it seems people are mostly in favour, with a few that reported
>>> being locked into SVN. I had anticipated that, and have proposed
>>> GitHub's SVN integration, which allows read-write access, so it should
>>> be mostly ok. We need more tests on that side to be sure, though.
>>> 
>>> The biggest problem we're facing right now is how to sync the repos.
>>> The existing llvm-repos format with all projects as sub-modules seem
>>> to be a good candidate, but I still haven't seen a consensus on how
>>> we'd do that.
>>> 
>>> About the migration plan, most people seem to agree a step-by-step
>>> process is necessary.
>> 
>> I personnally disagree with that, see below.
>> 
>>> So, first we move to git-only, possibly with
>>> sub-modules
>> 
>> If you move to git-only without the rest of the infrastructure/scripts, 
>> we're losing the convenience we have today with svn, and the "user 
>> experience" will not be so great. We may face resistance to this change.
>> I advocate to first set it up till it reaches the point of "good enough" in 
>> term of usability before actually migrating.
>> 
>>> , then we move to GitHub/Lab/BB,
>> 
>> It means we first need to host our git, write the tooling around it, to then 
>> migrate to github. I don't see the benefit over migrating directly to 
>> github: if people have to change their configuration, better have one single 
>> change.
>> 
>>> then we move Phab to
>>> GitHub merge requests, etc.
>> 
>> Phab to GitHub merge requests is a totally separate discussion IMO, and this 
>> discussion can happen after a successful move.
>> 
>> 
>>> 
>>> Regarding volunteers, I haven't yet asked around yet, but I'm sure we
>>> have enough interested people to help. If everything else fails, I'm
>>> more than happy to do all of that myself. Though, I'd have to learn a
>>> lot more about sub-modules than I know today, which is effectively
>>> nothing. :)
>> 
>> I'll be happy to throw manpower into tooling/infrastructure to make it 
>> happen.
>> 
>> -- 
>> Mehdi
>> 
>> ___
>> LLVM Developers mailing list
>> llvm-...@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.or

Re: [lldb-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Jacob Carlborg via lldb-dev

On 2016-06-01 13:17, Bruce Hoult via cfe-dev wrote:


I'm not sure why this isn't automatic with the
master repo commit checkout, but in any case it's not difficult.


Because there can be local changes that are not committed in the 
submodule. That can cause issues when switching to a different commit.


--
/Jacob Carlborg

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Dan Liew via lldb-dev
> Even though git 1.7 or something deals with submodules better - I
> don't personally like them at all. I'd rather have a convenience
> script or something which pulls and clones the sources. Not everyone
> needs all the sources and then there's the question of all the
> subprojects and build.. etc Not changing the workflow here would be
> most sane..

Git submodule don't force you to clone all the submodules. Until you run

```
git submodule init
git submodule update
```

the directories for each submodule will stay empty. Thus it isn't
necessary to pull down all the sources when using git submodules.
This would need support from the build system though. I'm not sure
what the build system would do right now if it found an empty
``tools/clang`` directory in the source tree.



> So clone llvm ; cd tools / ; clone ...
> --
> good luck with this.. I foresee near religious opinions on the horizon..

As do I.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [cfe-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Aaron Ballman via lldb-dev
On Wed, Jun 1, 2016 at 3:25 PM, James Y Knight  wrote:
> IMO, if we're switching to git, we should just be clear up front that all
> committers will be expected to switch to git as well -- or at least, if they
> want to use something else (e.g. mercurial's git bridge/etc), that it's
> their own problem.

So anyone still using svn should not expect it to work? That sounds
like a great way to alienate (at least some) active contributors.
However, I do agree that clarity would be nice regarding whether the
decision to switch to git has been "finalized" or not.

> It is truly NOT that big an imposition to require the use of git.

One thing to keep in mind is how well supported the tools are for the
platforms we have contributors actively developing on. As a data
point, I'm on Windows. Last time I tried using TortoiseGit (which
admittedly was over a year ago), it was not ready for production use
due to crashes with simple operations. On the other hand, TortoiseSVN
works very well. While I can certainly make use of the command line, I
don't predominately live in one like others might on non-Windows
systems. So yes, it may be an imposition to require the use of git.

I've not used the git integration in MSVC, so I can't speak to how
well that may work with a project as complex as ours (perhaps someone
else has experience and can speak to it), but that may also be a
viable alternative for those of us on Windows that are already using
MSVC. Other GUI alternatives may also exist that I'm simply unaware
of.

> And knowing how to use git at at least a basic level is an important skill 
> for a
> lot of software development now -- no matter what LLVM does, so I don't feel
> bad for making anyone spend time learning how to use it.
>
> I really don't think that promising and requiring that svn-client using
> people (especially committers: read-only access seems a lot less potentially
> problematic) will keep getting a good development experience after the
> migration is a good idea. I mean, if SVN also happens to work with the
> chosen hosting/workflow in the end, that's fine, I guess. But, I feel that
> should be considered a "if it works, that's okay, but it's not recommended,
> and is not guaranteed" kind of thing.

I'm uncomfortable with the idea of jettisoning SVN entirely right out
of the gate.

~Aaron

> Making that a requirement locks us into the use of github as the primary
> repository: no other git hosting has svn support, afaik.
> It means we can't introduce any workflows that wouldn't work well for svn
> users -- or if we do, that such users will probably complain anew when that
> happens.
> And if github's svn bridge turns out to have fatal problems, do we then
> abandon the migration?
>
>
>
> On Wed, Jun 1, 2016 at 2:36 PM, Aaron Ballman via cfe-dev
>  wrote:
>>
>> On Wed, Jun 1, 2016 at 2:18 PM, Renato Golin via llvm-dev
>>  wrote:
>> > On 1 June 2016 at 17:02, John Criswell  wrote:
>> >> Do you have a set of volunteers lined up to do such a migration?
>> >> Getting
>> >> people willing to do the migration will obviously be key, and that was
>> >> the
>> >> one thing I didn't see in the original email.
>> >
>> > Hi John,
>> >
>> > Well, first we need to know if people are in favour, then if the
>> > migration won't bring any serious problem, and then we can think of a
>> > migration plan. :)
>> >
>> > So far, it seems people are mostly in favour, with a few that reported
>> > being locked into SVN. I had anticipated that, and have proposed
>> > GitHub's SVN integration, which allows read-write access, so it should
>> > be mostly ok. We need more tests on that side to be sure, though.
>> >
>> > The biggest problem we're facing right now is how to sync the repos.
>> > The existing llvm-repos format with all projects as sub-modules seem
>> > to be a good candidate, but I still haven't seen a consensus on how
>> > we'd do that.
>> >
>> > About the migration plan, most people seem to agree a step-by-step
>> > process is necessary. So, first we move to git-only, possibly with
>> > sub-modules,
>>
>> Despite people's reservations of a git-only repository? I mean, we
>> still don't know that this will even work for people who wish to stay
>> with SVN. I am really not comfortable with this decision based on "it
>> should be mostly ok" from above, but maybe I am misunderstanding
>> something.
>>
>> ~Aaron
>> ___
>> cfe-dev mailing list
>> cfe-...@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Inquiry regarding AddOneMoreFrame function in UnWindLLDB

2016-06-01 Thread Jason Molenda via lldb-dev
It gets so tricky!  It's hard for the unwinder to tell the difference between a 
real valid stack unwind and random data giving lots of "frames".

It sounds like the problem that needs fixing is to figure out why the assembly 
unwind is wrong for frame 0.  What do you get for 

disass -a 

image show-unwind -a 

?


> On Jun 1, 2016, at 12:56 AM, Ravitheja Addepally  
> wrote:
> 
> Ok , currently the problem that I am facing is that there are cases in which 
> eh_frame should have been used for frame 0 but it isn't and the assembly 
> unwind just gives wrong information which could only be detected if the 
> debugger tried to extract more frames. Now the usage of AddOneMoreFrame in 
> UnwindLLDB is to try to get more than one frames in the stack. I want to run 
> both the unwinders and select the one that gives more number of frames.
> 
> On Wed, Jun 1, 2016 at 12:27 AM, Jason Molenda  wrote:
> 
> > On May 31, 2016, at 11:31 AM, jing...@apple.com wrote:
> >
> >
> >> On May 31, 2016, at 12:52 AM, Ravitheja Addepally via lldb-dev 
> >>  wrote:
> >>
> >> Hello,
> >>  I posted this query a while ago, i still have no answers, I am 
> >> currently working on Bug 27687 (PrintStackTraces), so the reason for the 
> >> failure is the erroneous unwinding of the frames from the zeroth frame. 
> >> The error is not detected in AddOneMoreFrame, since it only checks for 2 
> >> more frames, if it was checking more frames in AddOneMoreFrame, it would 
> >> have detected the error. Now my questions are ->
> >>
> >> ->  is that is there any specific reason for only checking 2 frames 
> >> instead of more ?
> >
> > The stepping machinery uses the unwinder on each stop to figure out whether 
> > it has stepped in or out, which is fairly performance sensitive, so we 
> > don't want AddOneMoreFrame to do more work than it has to.
> 
> 
> Most common case for a bad unwind, where the unwinder is stuck in a loop, is 
> a single stack frame repeating.  I've seen loops as much as six frames 
> repeating (which are not actually a series of recursive calls) but it's less 
> common.
> 
> >
> >> ->  Why no make the EH CFI based unwinder the default one and make the 
> >> assembly the fallback ?
> 
> 
> Sources of unwind information fall into two categories.  They can describe 
> the unwind state at every instruction of a function (asynchronous) or they 
> can describe the unwind state only at function call boundaries (synchronous).
> 
> Think of "asynchronous" here as the fact that the debugger can interrupt the 
> program at any point in time.
> 
> Most unwind information is designed for exception handling -- it is 
> synchronous, it can only throw an exception in the body of the function, or 
> an exception is passed up through it when it is calling another function.
> 
> For exception handling, there is no need/requirement to describe the prologue 
> or epilogue instructions, for instance.
> 
> eh_frame (and DWARF's debug_frame from which it derives) splits the 
> difference and makes things quite unclear.  It is guaranteed to be correct 
> for exception handling -- it is synchronous, and is valid in the middle of 
> the function and when it is calling other functions -- but it is a general 
> format that CAN be asynchronous if the emitter includes information about the 
> prologue or epilogue or mid-function stack changes.  But eh_frame is not 
> guaranteed to be that way, and in fact there's no way for it to indicate what 
> it describes, beyond the required unwind info for exception handling.
> 
> On x86, gcc and clang have always described the prologue unwind info in their 
> eh_frame.  gcc has recently started describing the epilogue too (clang does 
> not).  There's code in lldb (e.g. 
> UnwindAssembly_x86::AugmentUnwindPlanFromCallSite) written by Tong Shen when 
> interning at Google which will try to detect if the eh_frame describes the 
> prologue and epilogue.  If it does, it will use eh_frame for frame 0.  If it 
> only describes the prologue, it will use the instruction emulation code to 
> add epilogue instructions and use that at frame 0.
> 
> 
> There are other sources of unwind information similar to eh_frame that are 
> only for exception handling.  Tamas added ArmUnwindInfo last year which reads 
> the .ARM.exidx unwind tables.  I added compact unwind importing - an Apple 
> specific format that uses a single 4-byte word to describe the unwind state 
> for each function, which can't describe anything in the prologue/epilogue.  
> These formats definitely can't be used to unwind at frame 0 because we could 
> be stopped anywhere in the prologue/epilogue where they are not accurate.
> 
> 
> It's unfortunate that eh_frame doesn't include a way for the producer to 
> declare how async the unwind info is, it makes the debugger's job a lot more 
> difficult.
> 
> 
> J
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-d

Re: [lldb-dev] [cfe-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Renato Golin via lldb-dev
I think we should start two other threads: one about git tooling on Windows
and one about infrastructure problems migrating to git.

I'm confident we can solve both problems relatively easily.

Cheers,
Renato
On 1 Jun 2016 10:09 p.m., "Aaron Ballman"  wrote:

> On Wed, Jun 1, 2016 at 3:25 PM, James Y Knight 
> wrote:
> > IMO, if we're switching to git, we should just be clear up front that all
> > committers will be expected to switch to git as well -- or at least, if
> they
> > want to use something else (e.g. mercurial's git bridge/etc), that it's
> > their own problem.
>
> So anyone still using svn should not expect it to work? That sounds
> like a great way to alienate (at least some) active contributors.
> However, I do agree that clarity would be nice regarding whether the
> decision to switch to git has been "finalized" or not.
>
> > It is truly NOT that big an imposition to require the use of git.
>
> One thing to keep in mind is how well supported the tools are for the
> platforms we have contributors actively developing on. As a data
> point, I'm on Windows. Last time I tried using TortoiseGit (which
> admittedly was over a year ago), it was not ready for production use
> due to crashes with simple operations. On the other hand, TortoiseSVN
> works very well. While I can certainly make use of the command line, I
> don't predominately live in one like others might on non-Windows
> systems. So yes, it may be an imposition to require the use of git.
>
> I've not used the git integration in MSVC, so I can't speak to how
> well that may work with a project as complex as ours (perhaps someone
> else has experience and can speak to it), but that may also be a
> viable alternative for those of us on Windows that are already using
> MSVC. Other GUI alternatives may also exist that I'm simply unaware
> of.
>
> > And knowing how to use git at at least a basic level is an important
> skill for a
> > lot of software development now -- no matter what LLVM does, so I don't
> feel
> > bad for making anyone spend time learning how to use it.
> >
> > I really don't think that promising and requiring that svn-client using
> > people (especially committers: read-only access seems a lot less
> potentially
> > problematic) will keep getting a good development experience after the
> > migration is a good idea. I mean, if SVN also happens to work with the
> > chosen hosting/workflow in the end, that's fine, I guess. But, I feel
> that
> > should be considered a "if it works, that's okay, but it's not
> recommended,
> > and is not guaranteed" kind of thing.
>
> I'm uncomfortable with the idea of jettisoning SVN entirely right out
> of the gate.
>
> ~Aaron
>
> > Making that a requirement locks us into the use of github as the primary
> > repository: no other git hosting has svn support, afaik.
> > It means we can't introduce any workflows that wouldn't work well for svn
> > users -- or if we do, that such users will probably complain anew when
> that
> > happens.
> > And if github's svn bridge turns out to have fatal problems, do we then
> > abandon the migration?
> >
> >
> >
> > On Wed, Jun 1, 2016 at 2:36 PM, Aaron Ballman via cfe-dev
> >  wrote:
> >>
> >> On Wed, Jun 1, 2016 at 2:18 PM, Renato Golin via llvm-dev
> >>  wrote:
> >> > On 1 June 2016 at 17:02, John Criswell  wrote:
> >> >> Do you have a set of volunteers lined up to do such a migration?
> >> >> Getting
> >> >> people willing to do the migration will obviously be key, and that
> was
> >> >> the
> >> >> one thing I didn't see in the original email.
> >> >
> >> > Hi John,
> >> >
> >> > Well, first we need to know if people are in favour, then if the
> >> > migration won't bring any serious problem, and then we can think of a
> >> > migration plan. :)
> >> >
> >> > So far, it seems people are mostly in favour, with a few that reported
> >> > being locked into SVN. I had anticipated that, and have proposed
> >> > GitHub's SVN integration, which allows read-write access, so it should
> >> > be mostly ok. We need more tests on that side to be sure, though.
> >> >
> >> > The biggest problem we're facing right now is how to sync the repos.
> >> > The existing llvm-repos format with all projects as sub-modules seem
> >> > to be a good candidate, but I still haven't seen a consensus on how
> >> > we'd do that.
> >> >
> >> > About the migration plan, most people seem to agree a step-by-step
> >> > process is necessary. So, first we move to git-only, possibly with
> >> > sub-modules,
> >>
> >> Despite people's reservations of a git-only repository? I mean, we
> >> still don't know that this will even work for people who wish to stay
> >> with SVN. I am really not comfortable with this decision based on "it
> >> should be mostly ok" from above, but maybe I am misunderstanding
> >> something.
> >>
> >> ~Aaron
> >> ___
> >> cfe-dev mailing list
> >> cfe-...@lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-d

Re: [lldb-dev] [llvm-dev] [cfe-dev] GitHub anyone?

2016-06-01 Thread Piotr Padlewski via lldb-dev
2016-06-01 20:07 GMT+02:00 Anton Korobeynikov via llvm-dev <
llvm-...@lists.llvm.org>:

> >> Regarding the issue of git sub-modules and keeping Clang/LLVM in sync,
> perhaps we should just put Clang and LLVM into a single git repository and
> add a CMake option to disable compilation of Clang (the same could be done
> for other LLVM sub-projects for which bisection and other nifty features
> require a single revision number to refer to code across projects).
> Keeping these projects in separate repositories is just more work, and I
> don't see what we're getting out of that extra effort.
>
> I'm not sure we will benefit from having *all* the subprojects in the
> single repository. This might affect pull / clone time significantly.
> Think about having lld, lldb along with llvm + clang in the single
> main repo.
>
> --
> With best regards, Anton Korobeynikov
> Department of Statistical Modelling, Saint Petersburg State University
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

I also find it bad idea to have it all in one repo. Right now when I
develope clang-tidy, I don't have to recompile clang often because most of
the commits in clang-estra doesn't require new clang fetures.
This is pretty nice if work on 2 core machine.

Piotr
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [llvm-dev] GitHub anyone?

2016-06-01 Thread Josh Klontz via lldb-dev
(As a downstream user and very infrequent committer) +1 for a move to
GitHub. I'm indifferent about git, but pull requests significantly reduce
the effort of contributing small patches. I've found I'm much more likely
to submit a patch to a project if it is on GitHub.

On Tue, May 31, 2016 at 1:31 PM, Renato Golin via llvm-dev <
llvm-...@lists.llvm.org> wrote:

> Folks,
>
> There has been some discussion on IRC about SVN hosting and the perils
> of doing it ourselves. The consensus on the current discussion was
> that moving to a Git-only solution would have some disvantages, but
> many advantages. Furthermore, not hosting our own repos would save us
> a lot of headaches, admin costs and timed out connections.
>
> TL;DR: GitHub + git submodules [1] could replace all the functionality
> we have currently with SVN.
>
> (also GitLab, BitBucketc, etc).
>
> Here are some of the arguments made on IRC...
>
> 1. Due to SVN, we can't re-write history. If we use some GitHub
> properties [2], we could have the same effect.
>
> 2. Due to SVN, we have a mandatory time sequence, so commits go first
> in LLVM, then Clang (for example), and buildbots don't get lost. If we
> use submodules [1], we can have a similar relationship, but in a more
> explicit way, and the problem could be solved elegantly.
>
> 3. Some people still can only use SVN. For that, GitHub has an SVN
> interface [3] to the repositories.
>
> 4. We currently host our own SVN/Git, ViewVC and Klaus, Phabricator,
> etc. Not only this incurs in additional admin cost, but it also gets
> outdated, locally modified, and it needs to be backed up, etc. GitHub
> gives all that for us for free.
>
> 5. We can still use Bugzilla (and lock GitHub's own bug system), but
> we can also use GitHub's system to manage releases (it's actually
> quite good for that).
>
> 6. GitHub has automated testing of merge requests, meaning we can have
> pre-commit tests enabled on a set of fast bots, triggered by GitHub's
> own validation hooks. Even though that wouldn't cover everything,
> having a few pre-commit bots would considerably reduce the need to
> revert patches [citation needed].
>
> 7. With git submodules, we'd probably want to follow the same style we
> have today (llvm-projects/) instead of modelling how they look in
> tree (llvm/tools/clang still as a symlink).
>
> 8. Once we're solo Git, we can shop around *much* more easily. By
> using SVN, we're basically forced to host, or choose Source Forge.
> Using just Git, we can choose GitLab, BitBucket and many others, if
> GitHub is not appealing enough. Essentially, it doesn't matter where
> you are, the tools are good, there and largely replaceable [citation
> needed].
>
> What do people think? Any issue not covered that we should? How would
> that disrupt downstream users? Would it be a temporary disruption, but
> with long lasting benefits? Or will it just break everything for you?
>
> cheers,
> --renato
>
>
> [1] https://git-scm.com/book/en/v2/Git-Tools-Submodules
> [2]
> https://help.github.com/articles/defining-the-mergeability-of-pull-requests/
> [3] https://help.github.com/articles/support-for-subversion-clients/
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev