rengolin created this revision. rengolin added reviewers: lattner, chandlerc, jyknight, mehdi_amini, MatzeB, probinson, t.p.northover, chapuni, delcypher, dberlin, rsmith, beanz, cmatthews, asl, aaron.ballman, bcraig, Bigcheese, jroelofs, theraven, greened, hong.gyu.kim, rafael, AlexDenisov, silvas, friss, rovka, rnk. rengolin added subscribers: llvm-commits, cfe-commits, lldb-commits. rengolin set the repository for this revision to rL LLVM.
This is a draft on the proposal to moving to GitHub. Once we agree if this is a good proposal, we'll create the survey to get the final and complete intent of the community and act on it. *THIS IS NOT TO AGREE ON THE DECISION*, but only to agree on the proposal, which will then fuel the decision survey. Please, avoid arguments against Git or GitHub in this review. You'll have the opportunity to do so in the survey. Dates and times can be agreed later, but we're probably looking to a 6 month period anyway. I tried to include everyone I found in Phab that was on one of the discussions, but I may have missed a few. Feel free to add more people to it, but let's try to get a reasonable agreement to the proposal (not the move) under two weeks. Repository: rL LLVM https://reviews.llvm.org/D22463 Files: docs/Proposals/GitHub.rst
Index: docs/Proposals/GitHub.rst =================================================================== --- /dev/null +++ docs/Proposals/GitHub.rst @@ -0,0 +1,231 @@ +============================== +Moving LLVM Projects to GitHub +============================== + +Introduction +============ + +This is a proposal to move our current revision control system from Subversion +to GitHub. Below are the financial and technical arguments as to why we need +such a move and how will people (and validation infrastructure) continue to work +with a Git-based LLVM. + +There will be a survey pointing at this document when we'll know the community's +reaction and, if we collectively decide to move, the time-frames. Be sure to make +your views count. + +Essentially, the proposal is divided in the following parts: + + * Outline of the reasons to move to Git and GitHub + * Description on how the work flow will look like (compared to SVN) + * Remaining issues and potential problems + * The proposed migration plan + +Why Git, and Why GitHub? +======================== + +Why move at all? +---------------- + +The strongest reason for the move, and why this discussion started in the first +place, is that we currently host our own Subversion server and Git mirror in a +voluntary basis. The LLVM Foundation sponsors the server and provides limited +support, but there is only so much it can do. + +The volunteers are not Sysadmins themselves, but compiler engineers that happen +to know a thing or two about hosting servers. We also don't have 24/7 support, +and we sometimes wake up to see that continuous integration is broken because +the SVN server is either down or unresponsive. + +With time and money, the foundation and volunteers could improve our services, +implement more functionality and provide around the clock support, so that we +can have a first class infrastructure with which to work. But the cost is not +small, both in money and time invested. + +On the other hand, there are multiple services out there (GitHub, GitLab, +BitBucket among others) that offer that same service (24/7 stability, disk space, +Git server, code browsing, forking facilities, etc) for the very affordable price +of *FREE*. + +Why Git? +-------- + +Most new coders nowadays start with Git. A lot of them have never used SVN, CVS +or anything else. Websites like GitHub have changed the landscape of open source +contributions, reducing the cost of first contribution and fostering +collaboration. + +Git is also the version control most LLVM developers use. Despite the sources +being stored in an SVN server, most people develop using the Git-SVN integration, +and that shows that Git is not only more powerful than SVN, but people have +resorted to using a bridge because its features are now indispensable to their +internal and external workflows. + +In essence, Git allows you to: + * Commit, squash, merge, fork locally without any penalty to the server + * Add as many branches as necessary to allow for multiple threads of development + * Collaborate with peers directly, even without access to the Internet + * Have multiple trees without multiplying disk space, multiple concurrent builds + +In addition, because Git seems to be replacing every project's version control +system, there are many more tools that can use Git's enhanced feature set, so +new tooling is much more likely to support Git first (if not only), than any +other version control system. + +Why GitHub? +----------- + +GitHub, like GitLab and BitBucket, provide FREE code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that +we have today that serves code repository, mirroring, user control, etc. + +They also have a dedicated team to monitor, migrate, improve and distribute the +contents of the repositories depending on region and load. A level of quality +that we'd never have without spending money that would be better spent elsewhere, +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and quality in our community. + +GitHub has the added benefit that we already have a presence there. Many +developers use it already, and the mirror from our current repository is already +set up. + +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people stuck with SVN infrastructure and tooling can slowly migrate or +even stay working as if it was an SVN repository (including read-write access). + +So, any of the three solutions solve the cost and maintenance problem, but GitHub +has two additional features that would be beneficial to the migration plan as +well as the community already settled there. + + +How will the new workflow look like +=================================== + +In order to move version control, we need to make sure that we get all the +benefits with the least amount of problems. That's why the migration plan will +be slow, one step at a time, and we'll try to make it look as close as possible +to the current style without impacting the new features we want. + +As with the current SVN and Git repositories, each project will be separate, +on its own, and people will also be able to check them out in the same way +they're already doing today. + +In addition, we'll create a repository that will mimic our current *linear +history* repository. The most accepted proposal, then, was to have an umbrella +project that will contain *sub-modules* (https://git-scm.com/book/en/v2/Git-Tools-Submodules) +of all the LLVM projects and nothing else. + +This repository can be checked out on its own, in order to have *all* LLVM +projects in a single check-out, as many people have suggested, but it can also +only hold the references to the other projects, and be used for the sole purpose +of understanding the *sequence* in which commits were added by using the +``git rev-list --count hash`` or ``git describe hash`` commands. + +In a nutshell: + +* The projects' repositories will remain identical, with a new address (GitHub). +* They'll continue to have SVN RW access, but will also gain Git RW access. +* The linear history can still be accessed in the (RO) submodule meta project, + Which will continue to have SVN access. + +There is no need to additional tags, flags and properties, nor of external +services controlling the history, since both SVN and *git rev-list* can already +do that on their own. + +Essentially, we're adding Git RW access in addition to the already existing +structure, with all the additional benefits of it being in GitHub. + +Remaining questions and problems +================================ + +1. How much the SVN view emulates and how much it'll break tools/CI? + +For this one, we'll need people that will have problems in that area to tell +us what's wrong and how to help them fix it. + +We also recommend people and companies to migrate to Git, for its many other +additional benefits. + +2. How do we control access? + +Today we control access by adding people's keys on the SVN server. GitHub has a +nice key system, and in theory, it should be much easier to work with. + +But some people/companies might not be allowed to use GitHub or might have +firewalls blocking certain websites. + +3. How do we protect master from destruction? + +Today, SVN is reasonably stable and forcefully linear, but Git allows breaking +the order and merging branches, intentionally or not. This could break the origin +and cause havoc to everyone else, so we need a set of safe hooks and properties +that block anyone from upsetting the master. + +In GitHub speak, this means everyone can submit a pull request, but only a few +can merge them, but this still doesn't guarantee linear history, nor is fair +with the limited number of developers whose job will be to mainly merge +thousands of patches a day. + +4. Which tools will stop working? + +We know that LNT may break, since it relies on SVN's history. We can continue to +use LNT with the SVN-View, but it would be best to move it to Git once and for +all. + +Are there any other upstream systems that could be affected? + +Migration Plan +============== + +As soon as we decide to move, we'll have to set a date for the process to begin. + +As usual, we should be announcing big changes in one release to happen in the +next one. But since this won't impact external users (if they rely on our source +release tarballs), we don't necessarily have to. + +We will have to make sure all the *problems* reported are solved before the +final push. But we can start all non-binding processes (like mirroring to GitHub +and testing the SVN interface in it) before any hard decision. + +Here's a proposed plan: + +STEP #1 : Pre Move + +0. Update docs to mention the move, so people are aware the it's going on. +1. Register an official GitHub project with the LLVM foundation. +2. Setup another (read-only) mirror of llvm.org/git at this GitHub project. +3. Make sure we have an llvm-project (with submodules) setup in the official + account. +4. Make sure bisecting with llvm-project works. +5. Make sure no one has any other blocker. + +STEP #2 : Git Move + +6. Update the buildbots to pick up updates and commits from the official git + repository. +7. Update Phabricator to pick up commits from the official git repository. +8. Tell people living downstream to pick up commits from the official git + repository. +9. Give things time to settle. We could play some games like disabling the SVN + repository for a few hours on purpose so that people can test that their + infrastructure has really become independent of the SVN repository. + +Until this point nothing has changed for developers, it will just +boil down to a lot of work for buildbot and other infrastructure +owners. + +Once all dependencies are cleared, and all problems have been solved: + +STEP #3: Write Access Move + +10. Collect peoples GitHub account information, give them push access. Ideally + while still locking the GitHub repository somehow... +11. Switch SVN repository to read-only and allow pushes to the GitHub repository. +12. Mirror Git to SVN. + +STEP #4 : Post Move + +13. Archive the SVN repository, if GitHub's SVN is good enough. +14. Review and update *all* LLVM documentation. +15. Review website links pointing to viewvc/klaus/phab etc. to point to GitHub + instead.
_______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
