Re: Erratic Jenkins behavior

2015-02-18 Thread Colin P. McCabe
Hmm. I guess my thought would be that we would have a fixed number of "slots" (i.e. executors on a single node with associated .m2 directories). Then we wouldn't clear each .m2 in between runs, but we would ensure that only one slot at a time had access to each directory. In that case, build tim

Re: Erratic Jenkins behavior

2015-02-18 Thread Chris Nauroth
I¹m pretty sure there is no guarantee of isolation on a shared .m2/repository directory for multiple concurrent Maven processes. I¹ve had a theory for a while that one build running ³mvm install² can overwrite the snapshot artifact that was just installed by another concurrent build. This can cre

Re: Erratic Jenkins behavior

2015-02-12 Thread Colin P. McCabe
We could potentially use different .m2 directories for each executor. I think this has been brought up in the past as well. I'm not sure how maven handles concurrent access to the .m2 directory... if it's not using flock or fnctl then it's not really safe. This might explain some of our missing c

Re: Erratic Jenkins behavior

2015-02-10 Thread Steve Loughran
Mvn is a dark mystery to us all. I wouldn't trust it not pick up things from other builds if they ended up published to ~/.m2/repository during the process On 9 February 2015 at 19:29:06, Colin P. McCabe (cmcc...@apache.org) wrote: I'm sorry, I don't have any insigh

Re: Erratic Jenkins behavior

2015-02-09 Thread Arpit Agarwal
Sorry I don¹t have much insight into this either. Our Jenkins setup is a bit of a black box to me. Too few of us have access to the build hosts and it makes debugging difficult. Perhaps we should grant access to all committers. On 2/9/15, 11:29 AM, "Colin P. McCabe" wrote: >I'm sorry, I don't

Re: Erratic Jenkins behavior

2015-02-09 Thread Colin P. McCabe
I'm sorry, I don't have any insight into this. With regard to HADOOP-11084, I thought that $BUILD_URL would be unique for each concurrent build, which would prevent build artifacts from getting mixed up between jobs. Based on the value of PATCHPROCESS that Kihwal posted, perhaps this is not the c

Re: Erratic Jenkins behavior

2015-02-05 Thread Yongjun Zhang
Thanks Kihwal for bringing this up. Seems related to: https://issues.apache.org/jira/browse/HADOOP-11084 Hi Andrew/Arpit/Colin/Steve, you guys worked on this jira before, any insight about the issue Kihwal described? Thanks. --Yongjun On Thu, Feb 5, 2015 at 9:49 AM, Kihwal Lee wrote: > I a

Erratic Jenkins behavior

2015-02-05 Thread Kihwal Lee
I am sure many of us have seen strange jenkins behavior out of the precommit builds. - build artifacts missing - serving build artifact belonging to another build. This also causes wrong precommit results to be posted on the bug. - etc. The latest one I saw is disappearance of the unit test st