Re: Flaky Travis CI builds on master

2019-03-04 Thread Wes McKinney
hi Francois, I updated the CI failure page https://cwiki.apache.org/confluence/display/ARROW/Continuous+Integration+failures to split the resolved issues out into a separate filter. This makes things clearer and we can see now there are 7 open issues in the filter I also changed the language a bi

Re: Flaky Travis CI builds on master

2019-03-04 Thread Francois Saint-Jacques
Hello, I created a new label named `ci-failure`, which was retroactively applied to most issues triggering a CI failure in other PRs/master (I searched for travis-ci.org/apache/arrow and tagged them). The goal here is to track issues which generates false positives failure in PRs and ideally minim

Re: Flaky Travis CI builds on master

2019-03-03 Thread Kouhei Sutou
bbdc7ec932e3ba0f8a319ec2d1 plasma_store_server may have a signal handling bug. SIGTERM is ignored. Thanks, -- kou In <20190228.061800.1773971941530343560@clear-code.com> "Re: Flaky Travis CI builds on master" on Thu, 28 Feb 2019 06:18:00 +0900 (JST), Kouhei Sutou wrote: > Hi

Re: Flaky Travis CI builds on master

2019-03-02 Thread Wes McKinney
I just gave you edit access. If any PMC member would like to be an admin on the Confluence space (and you are not already), please let me know and I'll add you so you can help with the wiki admin requests On Fri, Mar 1, 2019 at 8:09 PM Francois Saint-Jacques wrote: > > Could someone give me writ

Re: Flaky Travis CI builds on master

2019-03-01 Thread Francois Saint-Jacques
Could someone give me write/edit access to confluence? Thank you, François On Fri, Mar 1, 2019 at 3:55 PM Francois Saint-Jacques < fsaintjacq...@gmail.com> wrote: > I'll take this. > > On Fri, Mar 1, 2019 at 3:55 PM Wes McKinney wrote: > >> We could create a page on the wiki that shows all open

Re: Flaky Travis CI builds on master

2019-03-01 Thread Francois Saint-Jacques
I'll take this. On Fri, Mar 1, 2019 at 3:55 PM Wes McKinney wrote: > We could create a page on the wiki that shows all open and resolved > issues relating to unexpected CI / build failures. Would someone like > to give this a go? There are probably many historical issues that can > be tagged wit

Re: Flaky Travis CI builds on master

2019-03-01 Thread Wes McKinney
We could create a page on the wiki that shows all open and resolved issues relating to unexpected CI / build failures. Would someone like to give this a go? There are probably many historical issues that can be tagged with the label On Fri, Mar 1, 2019 at 12:45 PM Francois Saint-Jacques wrote: >

Re: Flaky Travis CI builds on master

2019-03-01 Thread Francois Saint-Jacques
I agree with adding a tag/label for this and even marking the failure as critical. On Fri, Mar 1, 2019 at 12:18 PM Micah Kornfield wrote: > Moving away from the tactical for a minute, I think being able to track > these over time would be useful. I can think of a couple of high level > approac

Re: Flaky Travis CI builds on master

2019-03-01 Thread Micah Kornfield
Moving away from the tactical for a minute, I think being able to track these over time would be useful. I can think of a couple of high level approaches and I was wondering what others think. 1. Use tags appropriately in JIRA and try to generate a report from that. 2. Create a new confluence p

Re: Flaky Travis CI builds on master

2019-03-01 Thread Francois Saint-Jacques
Also just created https://issues.apache.org/jira/browse/ARROW-4728 On Thu, Feb 28, 2019 at 3:53 AM Ravindra Pindikura wrote: > > > > On Feb 28, 2019, at 2:10 PM, Antoine Pitrou wrote: > > > > > > Le 28/02/2019 à 07:53, Ravindra Pindikura a écrit : > >> > >> > >>> On Feb 27, 2019, at 1:48 AM, An

Re: Flaky Travis CI builds on master

2019-02-28 Thread Ravindra Pindikura
> On Feb 28, 2019, at 2:10 PM, Antoine Pitrou wrote: > > > Le 28/02/2019 à 07:53, Ravindra Pindikura a écrit : >> >> >>> On Feb 27, 2019, at 1:48 AM, Antoine Pitrou wrote: >>> >>> On Tue, 26 Feb 2019 13:39:08 -0600 >>> Wes McKinney wrote: hi folks, We haven't had a green b

Re: Flaky Travis CI builds on master

2019-02-28 Thread Antoine Pitrou
Le 28/02/2019 à 07:53, Ravindra Pindikura a écrit : > > >> On Feb 27, 2019, at 1:48 AM, Antoine Pitrou wrote: >> >> On Tue, 26 Feb 2019 13:39:08 -0600 >> Wes McKinney wrote: >>> hi folks, >>> >>> We haven't had a green build on master for about 5 days now (the last >>> one was February 21). H

Re: Flaky Travis CI builds on master

2019-02-27 Thread Ravindra Pindikura
> On Feb 27, 2019, at 1:48 AM, Antoine Pitrou wrote: > > On Tue, 26 Feb 2019 13:39:08 -0600 > Wes McKinney wrote: >> hi folks, >> >> We haven't had a green build on master for about 5 days now (the last >> one was February 21). Has anyone else been paying attention to this? >> It seems we sh

Re: Flaky Travis CI builds on master

2019-02-27 Thread Kouhei Sutou
Hi, In "Re: Flaky Travis CI builds on master" on Tue, 26 Feb 2019 16:00:31 -0600, Wes McKinney wrote: > * Seemingly a GLib Plasma OOM > https://travis-ci.org/apache/arrow/jobs/498906118#L3689 I take this: https://issues.apache.org/jira/browse/ARROW-4704 It seems that plas

Re: Flaky Travis CI builds on master

2019-02-27 Thread Brian Hulette
Another instance of #1 for the JS builds: https://travis-ci.org/apache/arrow/jobs/498967250#L992 I filed https://issues.apache.org/jira/browse/ARROW-4695 about it before seeing this thread. As noted there I was able to replicate the timeout on my laptop at least once. I didn't think to monitor mem

Re: Flaky Travis CI builds on master

2019-02-27 Thread Francois Saint-Jacques
I think we're witnessing multiple issues. 1. Travis seems to be slow (is it an OOM issue?) - https://travis-ci.org/apache/arrow/jobs/499122041#L1019 - https://travis-ci.org/apache/arrow/jobs/498906118#L3694 - https://travis-ci.org/apache/arrow/jobs/499146261#L2316 2. https://issues.apache.or

Re: Flaky Travis CI builds on master

2019-02-26 Thread Wes McKinney
Here's a build that just ran https://travis-ci.org/apache/arrow/builds/498906102?utm_source=github_status&utm_medium=notification 2 failed builds * ARROW-4684 * Seemingly a GLib Plasma OOM https://travis-ci.org/apache/arrow/jobs/498906118#L3689 24 hours ago: https://travis-ci.org/apache/arrow/

Re: Flaky Travis CI builds on master

2019-02-26 Thread Michael Sarahan
Yes, please let us know. We definitely see 500's from anaconda.org, though I'd expect less of them from CDN-enabled channels. On Tue, Feb 26, 2019 at 3:18 PM Uwe L. Korn wrote: > Hello Wes, > > if there are 500er errors it might be useful to report them somehow to > Anaconda. They recently migr

Re: Flaky Travis CI builds on master

2019-02-26 Thread Uwe L. Korn
Hello Wes, if there are 500er errors it might be useful to report them somehow to Anaconda. They recently migrated conda-forge to a CDN enabled account and this could be one of the results of that. Probably they need to still iron out some things. Uwe On Tue, Feb 26, 2019, at 8:40 PM, Wes McK

Re: Flaky Travis CI builds on master

2019-02-26 Thread Antoine Pitrou
On Tue, 26 Feb 2019 13:39:08 -0600 Wes McKinney wrote: > hi folks, > > We haven't had a green build on master for about 5 days now (the last > one was February 21). Has anyone else been paying attention to this? > It seems we should start cataloging which tests and build environments > are the mo

Flaky Travis CI builds on master

2019-02-26 Thread Wes McKinney
hi folks, We haven't had a green build on master for about 5 days now (the last one was February 21). Has anyone else been paying attention to this? It seems we should start cataloging which tests and build environments are the most flaky and see if there's anything we can do to reduce the flakine