Alan McKinnon <alan.mckinnon <at> gmail.com> writes:
> > often a custom, scheduler/framework has to be modified or custom developed, > > but I find it hard to believe there is no way to massively speed up > > something like gitlab on a robust linux cluster. That dragon has been > > sleighed for most all load problems, be it HPC, networking issues, or > > security audits, etc etc. Often, a cloud/cluster software can be radically > > sped up by allocating lots of extra ram to the framework it is running on. > > Cloud vendors charge way to much for extra ram, so performance of > > ram-intensive services are often run where there is ample ram. > Depends. It might all be running on a single-threaded ruby process in > the middleware If you believe what one reads:: "You seem to be confusing two very different things here: the Ruby Programming Language and the specific threading model of one specific implementation of the Ruby Programming Language. There are currently around 11 different implementations of the Ruby Programming Language, with very different and unique threading models. " circa 2011. > Don't laugh, I've seen stranger things. > Alan There are a multitude of ways to profile codes and the modules. Surely someone at gitlab has done this? (Evidently No?) A big part of cluster architecture is to identify the bottlenecks in codes that run on a single machine and implement solutions to lessen the bottlenecks. Most modern codes inherently run parallel and that's what cluster provide, more resources for many more threads, and the addition of custom hardware, such as GPUs, DDR5 ram, fpga and different architectures of processors. Surely gitlab can be fixed with a correctly applied cluster, or the codes that compose gitlab, need to be fixed... I'll be the first one to verify, you just don't rent a cluster (containers/VM) and move you slow applications there and magically all is fixed. The first step is to identify and qualify/quantify the source(s) of the bottlnecks. CS-101. However, profiling code/kernels/libs often leads one down the 'fuzzer' path, and that's a whole other (fun) time-sink. ymmv, James
