[hibernate-dev] Releases and CI setup
Hi, So, as expected, I'm not very happy with the new CI setup when doing releases. The issue is that each commit to ORM triggers at least 5 jobs (5.0, 5.1, 5,2, master-h2, master-check) which takes all the slave bandwidth we have. Note that I'm talking of ORM because it's where the issue is the most prominent but I'm pretty sure I would have the same issue with HV, considering we now have quite a few maintenance branches, except that the HV builds are faster. So I would say: - either we fix the issue we have with all the branches being tested for each commit that we discussed numerous times - or we need a mechanism to start specific slaves for releases - but I'm not sure it's fast enough tbh Because it's really annoying. Especially when you have to run your release job 3 times in a row due to SF.net issues... Releasing is already a tedious process, let's not make it even more tedious. (yeah I know it's not the first time I complain about it but it's really worse than it was before) -- Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Releases and CI setup
Then we should move to a CI server that is not erroneously triggering jobs it should not. On Mon, Apr 30, 2018 at 11:30 AM Guillaume Smet wrote: > Hi, > > So, as expected, I'm not very happy with the new CI setup when doing > releases. > > The issue is that each commit to ORM triggers at least 5 jobs (5.0, 5.1, > 5,2, master-h2, master-check) which takes all the slave bandwidth we have. > > Note that I'm talking of ORM because it's where the issue is the most > prominent but I'm pretty sure I would have the same issue with HV, > considering we now have quite a few maintenance branches, except that the > HV builds are faster. > > So I would say: > - either we fix the issue we have with all the branches being tested for > each commit that we discussed numerous times > - or we need a mechanism to start specific slaves for releases - but I'm > not sure it's fast enough tbh > > Because it's really annoying. Especially when you have to run your release > job 3 times in a row due to SF.net issues... > > Releasing is already a tedious process, let's not make it even more > tedious. > > (yeah I know it's not the first time I complain about it but it's really > worse than it was before) > > -- > Guillaume > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Releases and CI setup
Starting a new slave only takes 3 minutes, but I believe it has to be a "manual start" from its admin dashboard as Jenkins's scaling plugin is limited. Fixing the Jenkins triggers would be my preference. Alternatively: - we could look at pipelines - run all jobs within Docker -> improved isolation would allow us to run N builds per machine - combine these: run it all on Openshift On 30 April 2018 at 17:09, Guillaume Smet wrote: > Hi, > > So, as expected, I'm not very happy with the new CI setup when doing > releases. > > The issue is that each commit to ORM triggers at least 5 jobs (5.0, 5.1, > 5,2, master-h2, master-check) which takes all the slave bandwidth we have. > > Note that I'm talking of ORM because it's where the issue is the most > prominent but I'm pretty sure I would have the same issue with HV, > considering we now have quite a few maintenance branches, except that the > HV builds are faster. > > So I would say: > - either we fix the issue we have with all the branches being tested for > each commit that we discussed numerous times > - or we need a mechanism to start specific slaves for releases - but I'm > not sure it's fast enough tbh > > Because it's really annoying. Especially when you have to run your release > job 3 times in a row due to SF.net issues... > > Releasing is already a tedious process, let's not make it even more tedious. > > (yeah I know it's not the first time I complain about it but it's really > worse than it was before) > > -- > Guillaume > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Releases and CI setup
On Mon, Apr 30, 2018 at 8:34 PM, Sanne Grinovero wrote: > Starting a new slave only takes 3 minutes, but I believe it has to be > a "manual start" from its admin dashboard as Jenkins's scaling plugin > is limited. > > Fixing the Jenkins triggers would be my preference. > Yeah, last time we discussed this on HipChat, I have all the useful pointers. The code changes to the official plugin would be minimal. The only thing I'm worried about is how we would maintain this plugin. Alternatively: > - we could look at pipelines > How would they solve the issue? > - run all jobs within Docker -> improved isolation would allow us to > run N builds per machine > Would that help? I.e. are the machines we start powerful enough to run several jobs in parallel? I suspect, it wouldn't change the issue, just change the numbers of servers we would need (which might be good anyway but is not related to the issue at hand). -- Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Releases and CI setup
Using docker might be a nice idea if the machines are powerful enough. I will just mention it here but for the release only we can also not use Jenkins and run the command we need from the terminal of ci.hibernate.org. We already have the scripts ready so it shouldn't be too hard. If the Jenkins plugin doesn't work the way we need I don't feel like creating our own branch and I will consider it only if it's about sending a PR somewhere. But all this won't solve the problem with SourceForge that seems to be the main reason we see failures lately. On Mon, Apr 30, 2018 at 7:42 PM, Guillaume Smet wrote: > On Mon, Apr 30, 2018 at 8:34 PM, Sanne Grinovero > wrote: > >> Starting a new slave only takes 3 minutes, but I believe it has to be >> a "manual start" from its admin dashboard as Jenkins's scaling plugin >> is limited. >> >> Fixing the Jenkins triggers would be my preference. >> > > Yeah, last time we discussed this on HipChat, I have all the useful > pointers. The code changes to the official plugin would be minimal. The > only thing I'm worried about is how we would maintain this plugin. > > Alternatively: >> - we could look at pipelines >> > > How would they solve the issue? > > >> - run all jobs within Docker -> improved isolation would allow us to >> run N builds per machine >> > > Would that help? I.e. are the machines we start powerful enough to run > several jobs in parallel? > > I suspect, it wouldn't change the issue, just change the numbers of servers > we would need (which might be good anyway but is not related to the issue > at hand). > > -- > Guillaume > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
