Re: Gitlab Evaluation & Migration
Em 24/02/2019 22:25, Filipe Saraiva escreveu: > > 4. Is there any date to officially start the migration for all projects? > How will it be? > Will the migration happen before GSoC? Is it possible to a project request for migration and set Gitlab in the time for GSoC? -- Filipe Saraiva http://filipesaraiva.info/
Re: Gitlab Evaluation & Migration
On Thu, Feb 28, 2019 at 10:23 PM Filipe Saraiva wrote: > > Em 24/02/2019 22:25, Filipe Saraiva escreveu: > > > > 4. Is there any date to officially start the migration for all projects? > > How will it be? > > > > Will the migration happen before GSoC? Is it possible to a project > request for migration and set Gitlab in the time for GSoC? There is quite a bit of infrastructural work that will need to be done should the consensus conclusion be that we should be migrating. As such there is no guarantee that the migration will be in progress or complete by the time GSoC starts. Given we're still in early discussion on this, no policy around how projects not involved in the evaluation can opt in early has yet been made. > > -- > Filipe Saraiva > http://filipesaraiva.info/ Regards, Ben
Re: Gitlab Evaluation & Migration
On Wed, 27 Feb 2019 23:02:03 -0700 Ben Cooksley wrote > In terms of server load, it would be nice if the setup of forks was > still something the developer had to initiate rather than being done > automatically for every repository touched by kdesrc-build (I say this > mainly as if we had 50 people fork just half of the mainline > repositories we have, that's ~450GB of space used up - a massive > scalability issue) This seems like a challenge that needs to be addressed regardless of whether or not kdesrc-build does it automatically, because people creating tons and tons of forks is guaranteed to happen anyway if we move to Gitlab. It seems non-optimal if having more people able to submit merge requests results in the potential to blow up our servers. Nate
Re: Gitlab Evaluation & Migration
On Fri, Mar 1, 2019 at 3:13 AM Nate Graham wrote: > > On Wed, 27 Feb 2019 23:02:03 -0700 Ben Cooksley > wrote > > In terms of server load, it would be nice if the setup of forks was > > still something the developer had to initiate rather than being done > > automatically for every repository touched by kdesrc-build (I say this > > mainly as if we had 50 people fork just half of the mainline > > repositories we have, that's ~450GB of space used up - a massive > > scalability issue) > > This seems like a challenge that needs to be addressed regardless of whether > or not kdesrc-build does it automatically, because people creating tons and > tons of forks is guaranteed to happen anyway if we move to Gitlab. It seems > non-optimal if having more people able to submit merge requests results in > the potential to blow up our servers. We have a little over 1,000 mainline repositories, so in the above example we'd be talking about 25,000 forks being created - and i'd be expecting quite a bit more than 50 people to use kdesrc-build. To use another scenario, if the metric of half the repositories being involved (or even a quarter) held true with say 300 users, you're now looking at 75,000 - 150,000 forked repositories (and probably around 1.4TB - 2.7TB of space used) courtesy of an automated tool. It would take quite a while for us to reach 150,000 forked repositories on Gitlab if humans were to be creating these manually, however if an automated tool is going to be creating them as part of it's workflow, then we will hit it much more quickly (and is a phenomenal waste of resources given virtually all of those forks will never be utilised) I certainly do expect a number of forks to be created yes, but i'd rather they be useful forks where someone at least intends on working on something, rather than ones created automatically by software "just in case" someone decides to work on a project. > > Nate > Cheers, Ben
Re: Gitlab Evaluation & Migration
On Thu, Feb 28, 2019 at 9:43 PM Ben Cooksley wrote: > On Fri, Mar 1, 2019 at 3:13 AM Nate Graham wrote: > > > > On Wed, 27 Feb 2019 23:02:03 -0700 Ben Cooksley > wrote > > > In terms of server load, it would be nice if the setup of forks was > > > still something the developer had to initiate rather than being done > > > automatically for every repository touched by kdesrc-build (I say this > > > mainly as if we had 50 people fork just half of the mainline > > > repositories we have, that's ~450GB of space used up - a massive > > > scalability issue) > > > > This seems like a challenge that needs to be addressed regardless of > whether or not kdesrc-build does it automatically, because people creating > tons and tons of forks is guaranteed to happen anyway if we move to Gitlab. > It seems non-optimal if having more people able to submit merge requests > results in the potential to blow up our servers. > > We have a little over 1,000 mainline repositories, so in the above > example we'd be talking about 25,000 forks being created - and i'd be > expecting quite a bit more than 50 people to use kdesrc-build. To use > another scenario, if the metric of half the repositories being > involved (or even a quarter) held true with say 300 users, you're now > looking at 75,000 - 150,000 forked repositories (and probably around > 1.4TB - 2.7TB of space used) courtesy of an automated tool. > > It would take quite a while for us to reach 150,000 forked > repositories on Gitlab if humans were to be creating these manually, > however if an automated tool is going to be creating them as part of > it's workflow, then we will hit it much more quickly (and is a > phenomenal waste of resources given virtually all of those forks will > never be utilised) > I wonder if advanced filesystem features like ZFS deduplication may help in this situation. > I certainly do expect a number of forks to be created yes, but i'd > rather they be useful forks where someone at least intends on working > on something, rather than ones created automatically by software "just > in case" someone decides to work on a project. > > > > > Nate > > > > Cheers, > Ben >
Re: Gitlab Evaluation & Migration
On Fri, 1 Mar 2019, 07:21 Gleb Popov <6year...@gmail.com wrote: > > > On Thu, Feb 28, 2019 at 9:43 PM Ben Cooksley wrote: > >> On Fri, Mar 1, 2019 at 3:13 AM Nate Graham wrote: >> > >> > On Wed, 27 Feb 2019 23:02:03 -0700 Ben Cooksley < >> bcooks...@kde.org> wrote >> > > In terms of server load, it would be nice if the setup of forks was >> > > still something the developer had to initiate rather than being done >> > > automatically for every repository touched by kdesrc-build (I say >> this >> > > mainly as if we had 50 people fork just half of the mainline >> > > repositories we have, that's ~450GB of space used up - a massive >> > > scalability issue) >> > >> > This seems like a challenge that needs to be addressed regardless of >> whether or not kdesrc-build does it automatically, because people creating >> tons and tons of forks is guaranteed to happen anyway if we move to Gitlab. >> It seems non-optimal if having more people able to submit merge requests >> results in the potential to blow up our servers. >> >> We have a little over 1,000 mainline repositories, so in the above >> example we'd be talking about 25,000 forks being created - and i'd be >> expecting quite a bit more than 50 people to use kdesrc-build. To use >> another scenario, if the metric of half the repositories being >> involved (or even a quarter) held true with say 300 users, you're now >> looking at 75,000 - 150,000 forked repositories (and probably around >> 1.4TB - 2.7TB of space used) courtesy of an automated tool. >> >> It would take quite a while for us to reach 150,000 forked >> repositories on Gitlab if humans were to be creating these manually, >> however if an automated tool is going to be creating them as part of >> it's workflow, then we will hit it much more quickly (and is a >> phenomenal waste of resources given virtually all of those forks will >> never be utilised) >> > > I wonder if advanced filesystem features like ZFS deduplication may help > in this situation. > Deduplication can only do so much. Even if you were to solve the capacity/resource issues, you would hit another issue: Your personal namespace would be flooded with these automatically created forks. This would make your actual personal projects (which are future playground projects in most cases) nearly impossible to find. > >> I certainly do expect a number of forks to be created yes, but i'd >> rather they be useful forks where someone at least intends on working >> on something, rather than ones created automatically by software "just >> in case" someone decides to work on a project. >> >> > >> > Nate >> > >> >> Cheers, >> Ben >> > Regards, Ben >
Re: Gitlab Evaluation & Migration
Implementation details talk seems better suited for #kde-sysadmin :-) In any case, _automatically_ creating forks doesn't sound wise to me with my engineering hat on, because it's unnecessary. To create a fork, kdesrc-build would need authenticated API access. If it has has that, it can use it at any time. For example, if could have a command like this: $ kdesrc-build dev ... which then: - puts a nice commit template into the repo - asks the username / does API auth - uses the API to create the fork - adds the remote Again though: This is taking us far off-topic. Cheers, Eike
KDE Onboarding Sprint - Making it easy to setup a development environment
Hi everyone, cross-posting here from the kde-community mailing list. One of the major tasks for the Onboarding goal is to make it easier for newcomers to KDE to set up a development environment, so they can start working on the projects they would like to contribute in: https://phabricator.kde.org/T8484 We decided to organize a sprint to start working on this challenging task, in order to: - Identify our needs - Brainstorm available options regarding development environments - Decide on the most appropriate solution - Set the next steps in order to implement the chosen solution You will find much more info on the related Phabricator task: https://phabricator.kde.org/T8623 The Sprint will take place in Athens, Greece over a weekend of May 2019. The place that is hosting us will be available for May 4-5, 11-12 and 25-26. If you wish to attend, please fill in your availability on this poll so we can decide on a final date: https://framadate.org/kde-onboarding-sprint I've also just created a channel on matrix for the sprint: https://webchat.kde.org/#/room/#kde-onboarding-sprint:kde.org Look forward to welcoming you in Athens for a productive KDE sprint! Cheers, Neofytos
