Just for your information Greg - we are now experiencing another issue with
Travis CI. Suddenly our jobs are resource-strapped. Our builds started to
fail with Out of Memory errors and Not enough CPU (to run minikube). I
opened a critical infrastructure ticket -
https://issues.apache.org/jira/brows
Harsh "no" indeed - I understand where it comes from as I am looking from
my project perspective.
But maybe instead of binary yes/no answer - we can think about some
compromise/temporary solution (for a week or so while we run the release
and try to migrate out to another CI).
Maybe Increasing jus
On Mon, Jul 22, 2019 at 2:20 AM Jarek Potiuk
wrote:
> Hello Everyone (especially the infrastructure),
>
> Can we increase a number of workers/jobs we have per project now?
> Decreasing it to 5 (which I believe is the case) is terrible for us now
> We are nearing 1.10.4 release with Airflow and i
Thanks for looking and analysis :). Very correct.
However this is a special case actually. It only builds it that long
because we have a change in this branch that requires rebuilding more
layers of the image. In this case we add mysql dependencies in a different
way and we rebuild the whole stage
Jarek>Decreasing it to 5 (which I believe is the case) is terrible for us
now
Jarek>We are nearing 1.10.4 release with Airflow and if we have more than
one PR
Jarek>in the queue it waits for several hours to run!
I see 5 "pre-test" Travis jobs take 5-8 minutes each, and all the time is
basically
Hello Everyone (especially the infrastructure),
Can we increase a number of workers/jobs we have per project now?
Decreasing it to 5 (which I believe is the case) is terrible for us now
We are nearing 1.10.4 release with Airflow and if we have more than one PR
in the queue it waits for several ho
For reference, the Travis CI usage was also discussed in INFRA-18533.
Since July 8th the Flink project is no longer running PR builds on ASF
resources.Instead we mirror pull requests into an external repository
and run these builds in a sponsored Travis account. If anyone is
interested, check
Fantastic! but the link does not work for me :(
On Fri, Jul 19, 2019 at 8:16 AM Myrle Krantz wrote:
> Hey Jarek,
>
> Someone at Google was actually looking to get rid of about 11 GKE vouchers
> for $500 each; I don't think anyone has taken them up on that yet, if you
> need more.
>
>
> https://l
Hey Jarek,
Someone at Google was actually looking to get rid of about 11 GKE vouchers
for $500 each; I don't think anyone has taken them up on that yet, if you
need more.
https://lists.apache.org/thread.html/d9cb36493744836edcd59eebba6aef16356a3d5fd5fa90af336a82e1@%3Cmentors.community.apache.org%
OK. We got some credits from Google :) so I will soon be testing out
GitLabCI + GKE cluster combination and will let you know the results :)
J.
On Wed, Jul 17, 2019 at 7:38 AM Jarek Potiuk
wrote:
> From Apache Airflow side - I am going to try out GitLab CI setup for the
> project and I also rea
On Tue, Jul 16, 2019 at 5:37 PM Greg Stein wrote:
>
> Ray,
>
> Thanks for the offer of 50k minutes/project. That will definitely work for
> most projects.
>
> While we don't have precise measurements, some projects used *way* more
> than that within Travis last month:
>
> flink: 350k minutes
> arr
>From Apache Airflow side - I am going to try out GitLab CI setup for the
project and I also reached out to Google OSS team to donate a GKE
auto-scaling cluster for our workloads. If it works (highly probable), we
might be able to use even less of the free minutes from GitLab because we
will have t
Ray,
Thanks for the offer of 50k minutes/project. That will definitely work for
most projects.
While we don't have precise measurements, some projects used *way* more
than that within Travis last month:
flink: 350k minutes
arrow: 260k minutes
cloudstack: 190k minutes
incubator-druid: 96k
airflow
Joan,
The 50,000 minutes would be for each project (assuming individual projects
will apply for separate GitLab licenses).
When you reach the limit, you'll have an option to purchase additional
minutes. More info. available at
https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integra
Hi Raymond,
Would this 50,000 CI minutes per month be spread across the entire ASF,
or just each project? With >300 projects here, that's potentially
50,000 * 300 = 15 million minutes we're talking about.
What happens when a project exceeds that amount of minutes? Busy
projects that build each PR
Jarek,
You're not required to migrate your repo over to GitLab. We have other
projects that keep their source code in GitHub, but are using GitLab for
CI. Hope this helps...
Thanks,
Ray
On Tue, Jul 16, 2019 at 12:51 PM Jarek Potiuk
wrote:
> Yep we use Git indeed but we have Github repo (
> ht
Yep we use Git indeed but we have Github repo (
https://github.com/apache/airflow) and I believe this is pretty much
standard for all Apache projects (adding Greg as well).
I don't think (or am I wrong?) the open source program directly applies in
this case because we would have to have GitLab Re
Thanks Jarek:
We do have an open source program at GitLab (
https://about.gitlab.com/solutions/open-source/) where open source
projects get access to top tier features (either SaaS or self-hosted) for
free including up to 50,000 CI minutes/month.
Are you currently using Git as your source code r
Adding Raymond Paik who is GitLab Community Manager and wants to chime-in
the thread!
J.
On Thu, Jul 4, 2019 at 1:04 AM Allen Wittenauer
wrote:
>
>
> > On Jul 3, 2019, at 3:15 PM, Joan Touzet wrote:
> >
> > I was asking if any of the service platforms provided this. So far, it
> looks like no.
> On Jul 3, 2019, at 3:15 PM, Joan Touzet wrote:
>
> I was asking if any of the service platforms provided this. So far, it looks
> like no.
I was playing around bit with Drone today because we actually need ARM
in $DAYJOB and this convo reminded me that I needed to check it out.
On 2019-07-03 5:57 p.m., Greg Stein wrote:
On Wed, Jul 3, 2019 at 11:36 AM Allen Wittenauer
wrote:
...
CouchDB keeps receiving a lot of pressure to build on aarch64, ppc64le
and s390x, which keeps pushing us back to Jenkins CI (ASF or
independent). And if we have to do that, then not much
On Wed, Jul 3, 2019 at 11:36 AM Allen Wittenauer
wrote:
>...
> > CouchDB keeps receiving a lot of pressure to build on aarch64, ppc64le
> > and s390x, which keeps pushing us back to Jenkins CI (ASF or
> > independent). And if we have to do that, then not much else matters to
> us.
>
> One
> On Jul 3, 2019, at 8:04 AM, Joan Touzet wrote:
>
> (With my CouchDB release engineer hat on only)
>
> Anyone know if any of these external services supports platforms other
> than amd64/x86_64?
AFAIK, all of the commercial SaaS companies who have an "open source
can use for free”
(With my CouchDB release engineer hat on only)
Anyone know if any of these external services supports platforms other
than amd64/x86_64?
CouchDB keeps receiving a lot of pressure to build on aarch64, ppc64le
and s390x, which keeps pushing us back to Jenkins CI (ASF or
independent). And if we have
[builds@ to bcc: ; add vp-infra]
Jarek, Kamil ... that would be very interesting indeed. We've spoken to
Gitlab on possibilities in the past; it sounds like some of our projects
are interested in their services.
Please feel free to email myself/"us" at vp-infra@apache to discuss any
possible opti
I spoke to Kamil - Gitlab CI maintainer (in CC:) and he will speak to CEO
of GitLab and Product Managers of GitLab CI whether GitLab will be willing
to help with it.
J.
On Wed, Jul 3, 2019 at 9:33 AM Jarek Potiuk
wrote:
> Actually speaking of Gitlab CI. I realised my close friend is actually TH
Actually speaking of Gitlab CI. I realised my close friend is actually THE
maintainer and main person responsible for Gitlab CI. I will reach out to
him and see if they can help with this and provide free service. Shame I
have not thought about it before.
J.
On Wed, Jul 3, 2019 at 8:37 AM Allen W
> On Jul 2, 2019, at 11:12 PM, Jeff MAURY wrote:
>
> Azure pipeline vas the big plus of supporting Linux Windows and macos nodes
There’s a few that support various combinations of non-Linux. Gitlab
CI has been there for a while. Circle CI has had OS X and is in beta with
Windows.
Azure pipeline vas the big plus of supporting Linux Windows and macos nodes
And i think you can add you nodes to the pools
Jeff
Le mer. 3 juil. 2019 à 08:04, Allen Wittenauer
a écrit :
>
> > On Jul 2, 2019, at 10:21 PM, Greg Stein wrote:
> >
> > We'll keep this list apprised of anything we fin
> On Jul 2, 2019, at 10:21 PM, Greg Stein wrote:
>
> We'll keep this list apprised of anything we find. If anybody knows of,
> and/or can recommend a similar type of outsourced build service ... we
> *absolutely* would welcome pointers.
FWIW, we’ve been collecting them bit by bit into
On Tue, Jul 2, 2019 at 11:56 PM Jarek Potiuk wrote:
>...
> In the meantime maybe INFRA can help to coordinate some effort between
> Flink/Arrow/Airflow to decrease pressure on Travis? We considered few
> options (and are going to implement some of them shortly I think). Some of
> them are not dir
We also experience huge delays for Airflow (seems that we are the third "whale"
according to
https://lists.apache.org/thread.html/af52e2a3e865c01596d46374e8b294f2740587dbd59d85e132429b6c@%3Cbuilds.apache.org%3E)
We are evaluating other options for funding as well (including getting some
credi
Hi folks,
As this seems to be a hot topic as of late, I'll provide some
information about our usage of external CI services.
Travis CI: The foundation has an agreement with Travis CI to provide our
projects with external CI services through them. At current, we have
approximately 40 executors the
33 matches
Mail list logo
