On Thu, Feb 28, 2019 at 9:43 PM Ben Cooksley <bcooks...@kde.org> wrote:
> On Fri, Mar 1, 2019 at 3:13 AM Nate Graham <n...@kde.org> wrote: > > > > ---- On Wed, 27 Feb 2019 23:02:03 -0700 Ben Cooksley <bcooks...@kde.org> > wrote ---- > > > In terms of server load, it would be nice if the setup of forks was > > > still something the developer had to initiate rather than being done > > > automatically for every repository touched by kdesrc-build (I say this > > > mainly as if we had 50 people fork just half of the mainline > > > repositories we have, that's ~450GB of space used up - a massive > > > scalability issue) > > > > This seems like a challenge that needs to be addressed regardless of > whether or not kdesrc-build does it automatically, because people creating > tons and tons of forks is guaranteed to happen anyway if we move to Gitlab. > It seems non-optimal if having more people able to submit merge requests > results in the potential to blow up our servers. > > We have a little over 1,000 mainline repositories, so in the above > example we'd be talking about 25,000 forks being created - and i'd be > expecting quite a bit more than 50 people to use kdesrc-build. To use > another scenario, if the metric of half the repositories being > involved (or even a quarter) held true with say 300 users, you're now > looking at 75,000 - 150,000 forked repositories (and probably around > 1.4TB - 2.7TB of space used) courtesy of an automated tool. > > It would take quite a while for us to reach 150,000 forked > repositories on Gitlab if humans were to be creating these manually, > however if an automated tool is going to be creating them as part of > it's workflow, then we will hit it much more quickly (and is a > phenomenal waste of resources given virtually all of those forks will > never be utilised) > I wonder if advanced filesystem features like ZFS deduplication may help in this situation. > I certainly do expect a number of forks to be created yes, but i'd > rather they be useful forks where someone at least intends on working > on something, rather than ones created automatically by software "just > in case" someone decides to work on a project. > > > > > Nate > > > > Cheers, > Ben >
