Re: [build system] currently experiencing git timeouts when building

2018-01-18 Thread shane knapp
quick update: it looks like the timeouts have stopped. github finally got back to me about this, but (again) after they stopped happening. i'll be keeping an eye on this for the next few days and will re-escalate if we start having them again. shane On Tue, Jan 16, 2018 at 1:18 PM, shane knapp

Re: [build system] currently experiencing git timeouts when building

2018-01-16 Thread shane knapp
update: we're seeing about 3% of builds timing out, but today is (so far) a bad one: i've reached out to github about this and am waiting to hear back. $ get_timeouts.py 10 Timeouts by project: 17spark-branch-2.3-lint 10spark-branch-2.3-test-maven-hadoop-2.6 7spark-branch-2.3-comp

[build system] currently experiencing git timeouts when building

2018-01-10 Thread shane knapp
i just noticed we're starting to see the once-yearly rash of git timeouts when building. i'll be looking in to this today... i'm at our lab retreat, so my attention will be divided during the day but i will report back here once i have some more information. in the meantime, if your jobs have a