On Sat, Feb 22, 2025 at 05:24:05PM -0700, Simon Glass wrote: > Hi Tom, > > On Sat, 22 Feb 2025 at 14:37, Tom Rini <tr...@konsulko.com> wrote: > > > > On Sat, Feb 22, 2025 at 10:23:59AM -0700, Simon Glass wrote: > > > Hi Tom, > > > > > > On Fri, 21 Feb 2025 at 17:08, Tom Rini <tr...@konsulko.com> wrote: > > > > > > > > On Fri, Feb 21, 2025 at 04:42:09PM -0700, Simon Glass wrote: > > > > > Hi Tom, > > > > > > > > > > On Mon, 17 Feb 2025 at 07:14, Tom Rini <tr...@konsulko.com> wrote: > > > > > > > > > > > > On Mon, Feb 17, 2025 at 06:14:06AM -0700, Simon Glass wrote: > > > > > > > Hi Tom, > > > > > > > > > > > > > > On Sun, 16 Feb 2025 at 14:52, Tom Rini <tr...@konsulko.com> wrote: > > > > > > > > > > > > > > > > On Sun, Feb 16, 2025 at 12:39:34PM -0700, Simon Glass wrote: > > > > > > > > > Hi Tom, > > > > > > > > > > > > > > > > > > On Sun, 16 Feb 2025 at 09:07, Tom Rini <tr...@konsulko.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On Sun, Feb 16, 2025 at 07:10:12AM -0700, Simon Glass wrote: > > > > > > > > > > > Hi Tom, > > > > > > > > > > > > > > > > > > > > > > On Sat, 15 Feb 2025 at 11:12, Tom Rini > > > > > > > > > > > <tr...@konsulko.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Feb 15, 2025 at 10:21:16AM -0700, Simon Glass > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Hi Tom, > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, 15 Feb 2025 at 07:41, Tom Rini > > > > > > > > > > > > > <tr...@konsulko.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Feb 15, 2025 at 04:59:40AM -0700, Simon > > > > > > > > > > > > > > Glass wrote: > > > > > > > > > > > > > > > Hi Tom, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, 10 Feb 2025 at 09:25, Tom Rini > > > > > > > > > > > > > > > <tr...@konsulko.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Feb 06, 2025 at 03:38:55PM -0700, Simon > > > > > > > > > > > > > > > > Glass wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is a global default, so put it under > > > > > > > > > > > > > > > > > 'default' like the tags. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Signed-off-by: Simon Glass <s...@chromium.org> > > > > > > > > > > > > > > > > > Suggested-by: Tom Rini <tr...@konsulko.com> > > > > > > > > > > > > > > > > > Reviewed-by: Tom Rini <tr...@konsulko.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please make v4 include the way you redid the > > > > > > > > > > > > > > > > second patch and be on top > > > > > > > > > > > > > > > > of mainline, thanks. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > That's enough versions for me, so I'll let you do > > > > > > > > > > > > > > > that, if you'd like. > > > > > > > > > > > > > > > It probably doesn't affect your tree as not as > > > > > > > > > > > > > > > much is done in > > > > > > > > > > > > > > > parallel. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am disappointed. > > > > > > > > > > > > > > > > > > > > > > > > > > I'm sorry to disappoint you. > > > > > > > > > > > > > > > > > > > > > > > > > > The background is that I looked at the difference > > > > > > > > > > > > > between our trees > > > > > > > > > > > > > and the gitlab files are quite different. My CI runs > > > > > > > > > > > > > take about 35 > > > > > > > > > > > > > mins and it seems that yours is around 90 mins. I > > > > > > > > > > > > > would like to reduce > > > > > > > > > > > > > / remove the delta (for time and patch diff), but I'm > > > > > > > > > > > > > not sure how. > > > > > > > > > > > > > > > > > > > > > > > > > > My goal is to get CI runs to below 20 minutes, best > > > > > > > > > > > > > case. > > > > > > > > > > > > > > > > > > > > > > > > I'm sure CI could be quicker still with a number of > > > > > > > > > > > > faster runners. But > > > > > > > > > > > > if you can't be bothered to make changes against > > > > > > > > > > > > mainline, what is the > > > > > > > > > > > > point? > > > > > > > > > > > > > > > > > > > > > > If you recall, I was working with your tree and had > > > > > > > > > > > various ideas to > > > > > > > > > > > speed things up, but you didn't like it. So I've had to > > > > > > > > > > > do it in my > > > > > > > > > > > tree. This is not about more runners (although I might > > > > > > > > > > > have another > > > > > > > > > > > one soon). It is about running jobs in parallel. > > > > > > > > > > > > > > > > > > > > And I wasn't sure more runners in parallel would help (as > > > > > > > > > > it would slow > > > > > > > > > > down the fast runner which is what keeps the long jobs from > > > > > > > > > > being even > > > > > > > > > > longer) as much as adding more regular runners would (which > > > > > > > > > > we've done) > > > > > > > > > > and noted that in the end it's a configuration on the > > > > > > > > > > runner side so to > > > > > > > > > > go ahead. And I reviewed and ack'd the patches here which > > > > > > > > > > exposed the > > > > > > > > > > issues your path revealed. I just can't apply them because > > > > > > > > > > they need to > > > > > > > > > > be rebased (and squashed). > > > > > > > > > > > > > > > > > > You have already added tags for things, but (IIUC) they are > > > > > > > > > around the > > > > > > > > > other way from what I have added. > > > > > > > > > > > > > > > > > > I have a tag called 'single' which means that the machine is > > > > > > > > > only > > > > > > > > > allowed to one of those jobs. The world-build jobs are marked > > > > > > > > > with > > > > > > > > > 'single'. > > > > > > > > > > > > > > > > > > For other jobs, I allow the runners to pick up some in > > > > > > > > > parallel > > > > > > > > > depending on their performance (for moa and tui that is 10). > > > > > > > > > > > > > > > > > > So at most, there is a 'world build' and 10 test.py jobs > > > > > > > > > running on > > > > > > > > > the same machine. It seems to work fine in practice, although > > > > > > > > > I would > > > > > > > > > rather be able to make these two types of jobs mutually > > > > > > > > > exclusive, so > > > > > > > > > that a runner is either running 10 parallel jobs or 1 > > > > > > > > > 'single' job, > > > > > > > > > but not both. I'm not sure how to do that. > > > > > > > > > > > > > > > > So unless I'm missing something, in both cases the bottleneck > > > > > > > > is that > > > > > > > > for world build jobs you don't want anything else going on with > > > > > > > > the > > > > > > > > underlying build host. You could register 10 "all" runners and > > > > > > > > 1 "fast > > > > > > > > amd64" runner (and something similar but smaller for > > > > > > > > alexandra). If you > > > > > > > > update the registrations on source.denx.de can you then shut > > > > > > > > down your > > > > > > > > gitlab instance? > > > > > > > > > > > > > > I've put a tag of 'single' on things that should run on the > > > > > > > single-job > > > > > > > runner. Everything else can run concurrently, e.g. up to 10 jobs. > > > > > > > So I > > > > > > > have two runners on the same host. E.g. tui-single has 'limit = > > > > > > > 1', > > > > > > > but 'tui' has no limit and is just governed by the 'concurrent = > > > > > > > 10' > > > > > > > at the top of the file. > > > > > > > > > > > > Yes. And you could move those runners to the mainline gitlab. There > > > > > > is > > > > > > no "single" tag, that would be the "all" tag. And "tui-single" > > > > > > would be > > > > > > "fast amd64". > > > > > > > > > > They are still attached to the Denx gitlab. Nothing has changed on my > > > > > side. I'm not sure that your new tags are working though. I have a > > > > > feeling something broke along the way when you made all your tag > > > > > changes. One of my servers makes a bit of noise and I haven't heard it > > > > > in quite a while. > > > > > > > > There's a few of your runners that are "stale" and haven't contacted > > > > gitlab in a long time. I'll double check the tags tho. > > > > > > > > > If Denx would like to give me access to their gitlab instances, I'd be > > > > > happy to play around and figure out how to get it going as fast as my > > > > > tree does, and send a patch. > > > > > > > > I'm not sure what you mean by that? The instance itself? > > > > > > Yes. I can fiddle with tags on my runners and try to figure it out. > > > > I'm not sure what you're getting at here. If you mean "tags" in > > /etc/gitlab-runner/config.toml those aren't relevant here I believe. > > No, I mean the tags in CI. If I fiddle with them I can probably come > up with a way to run your CI much faster. Mine is about 35mins.
I'm not so sure about that. Yours runs faster because it tests less. Now that we've got some of your other fast runners showing up again, this is more instructive of current times I think: https://source.denx.de/u-boot/u-boot/-/pipelines/24802 If you want to make mainline CI run faster you will need to catch up with the missing coverage or argue that some things are redundant. > > > > > I also have another runner to add. > > > > > > > > I'll contact you off-list with the token. > > > > > > > > > > > From my side, I have found it helpful and refreshing to have a > > > > > > > gitlab > > > > > > > instance which I can control, e.g. it runs in half the time and > > > > > > > if my > > > > > > > patches are completely blocked by Linaro, etc., I have an escape > > > > > > > valve. > > > > > > > > > > > > Yes, and I have no idea what any of that has to do with anything > > > > > > other > > > > > > than leading to confusion about what tree is or is not mainline. > > > > > > Since > > > > > > you own u-boot.org and ci.u-boot.org is your gitlab and > > > > > > https://ci.u-boot.org/u-boot/u-boot/ is your personal tree. > > > > > > > > > > For now I am working with my tree, so that I am not blocked by Linaro, > > > > > etc. but as you have seen I can rebase series for your tree as needed. > > > > > > > > And you're not addressing my point about using the project domain for > > > > your personal tree. That's my big huge "are you forking the project or > > > > what" problem. > > > > > > I'm just making sure that my work is not blocked or lost, as that has > > > happened too many times in the past few years. > > > > Again, are you intending to fork the project? Putting your personal tree > > in as "https://ci.u-boot.org/u-boot/u-boot.git"; is not OK. I keep asking > > you to stop it. > > No, I'm not intending to fork anything. But I need a tree that I can > control and push things into. I don't know how you can call your personal tree being at "https://ci.u-boot.org/u-boot/u-boot.git"; and saying it's somewhere you control and can push to while not also saying it's a fork. If you want to close down your gitlab and CNAME ci.u-boot.org to source.denx.de, you can still push things to u-boot-dm. Or if that's too constrained of a namespace you can also get a contributors/sjg/ namespace. But what you're doing today WILL lead to confusion. > > > BTW weka is too slow so I took it off. > > > > I'll remove it. > > > > > tui is running but seems to be renamed 'tui-redux' now. I don't see it > > > getting jobs though. > > > > It's been getting jobs. Is it something sufficiently fast that it should > > be doing big jobs? > > Yes, tui is the same as moa, pretty fast. Thanks, I've added the "fast amd64" tag then so it'll get the long running jobs too. -- Tom
signature.asc
Description: PGP signature
