On Thu, Aug 18, 2016 at 5:29 PM, Uwe Schindler <uschind...@apache.org> wrote:
> Ah, > > > > you can delete the backup folder of old jenkins home-dir. > Done. > I nuked the workspaces inside already (after sudo jenkins, otherwise we > would have gone out of disk space), but I cannot remove the folder on its > own. > > > > The lucene.build.properties file is now the only “customization” of this > machine, but this one is hardware-specific just for this node. > > > ok, I'm happy for you to manage that yourselves. It is possible to maintain it in puppet but would be more lines of code than whats in that one file ;) Gav... > Uwe > > > > ----- > > Uwe Schindler > > uschind...@apache.org > > ASF Member, Apache Lucene PMC / Committer > > Bremen, Germany > > http://lucene.apache.org/ > > > > *From:* Gav [mailto:gmcdon...@apache.org] > *Sent:* Thursday, August 18, 2016 6:47 AM > *To:* Uwe Schindler <uschind...@apache.org> > *Cc:* d...@lucene.apache.org; builds <builds@apache.org> > > *Subject:* Re: JDK, Maven, Ant versions have been consolidated. > > > > Its fine where it is. > > Is there any more to do here for infra Uwe or is the Lucene Jenkins node > all good now ? > > Gav... > > > > On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <uschind...@apache.org> > wrote: > > Hi, > > > > it is not part of any distribution or in any developer checkouts. We just > placed it there for Jenkins access to run more extensive tests – we can > also move it to a private subversion branch, not accessible from public? We > could then add the Jenkins user to that folder. > > > > Uwe > > > > ----- > > Uwe Schindler > > uschind...@apache.org > > ASF Member, Apache Lucene PMC / Committer > > Bremen, Germany > > http://lucene.apache.org/ > > > > *From:* Sean Busbey [mailto:bus...@cloudera.com] > *Sent:* Wednesday, August 17, 2016 1:40 PM > *To:* builds <builds@apache.org> > *Cc:* gmcdon...@apache.org; d...@lucene.apache.org > > > *Subject:* RE: JDK, Maven, Ant versions have been consolidated. > > > > Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I > thought those licenses weren't okay to have in an ASF source repo? > > -- > Sean Busbey > > I set this up and configured Jenkins to use the test-data checked out from > Subversion. > > Uwe > > ----- > Uwe Schindler > uschind...@apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > > > From: Steve Rowe [mailto:sar...@gmail.com] > > Sent: Monday, August 15, 2016 8:40 PM > > To: d...@lucene.apache.org > > Subject: Re: JDK, Maven, Ant versions have been consolidated. > > > > Thanks Uwe for moving this process along. > > > > +1 to make a new SVN dir at https://svn.apache.org/repos/ > asf/lucene/test- > > data/ and put the enwiki data file there. > > > > -- > > Steve > > www.lucidworks.com > > > > > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <uschind...@apache.org> > > wrote: > > > > > > Hi, > > > > > > I thought about the wikipedia test files used by Jenkins: I think I > would > > commit them to SVN inside the Lucene/Solr project’s folder. The > Nightly-Test > > Jenkins jobs that use them could simply check them out into a separate > dir of > > the workspace. Byt that it is easier for us to update them. If you also > think > > this is fine, we can leave it like that. > > > > > > The ~jenkins/lucene.build.properties file is (as said before) > > machine/hardware specific. I’d keep them in Jenkins’ home dir. We may > > create a puppet module out of it, but this would prevent us from > optimizing > > or changing them quickly per project requirements. > > > > > > Finally the “ant ivy-bootstrap” task was for now separated into a > Jenkins > > job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy- > > Bootstrap/). In the future, we can add the “ivy-bootstrap” target as > > dependency to the “ant jenkins” and other targets which are solely > triggered > > by Jenkins, so separately bootstrapping is no longer required (please > keep in > > mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside > > workspace – this is why it’s done separately!). I just have to fix the > Jenkins- > > specific targets in our build.xml in Git repo to only do the actual > > bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ > machine. > > Then it’s a no-op on most builds. For now the separate Jenkins job is > enough > > to quickly do the bootstrap as workaround. > > > > > > If nobody stops me from committing the following file into SVN: > > > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh > > > total 2.9G > > > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt > > > > > > I will do this later this evening and change the Jenkins builds to do > an extra > > checkout into the workspace next to Git. > > > I’d suggest to place the folder contents of ~/lucene-data here: > > https://svn.apache.org/repos/asf/lucene/test-data > > > > > > Uwe > > > > > > ----- > > > Uwe Schindler > > > uschind...@apache.org > > > ASF Member, Apache Lucene PMC / Committer > > > Bremen, Germany > > > http://lucene.apache.org/ > > > > > > From: Uwe Schindler [mailto:uschind...@apache.org] > > > Sent: Monday, August 15, 2016 12:47 PM > > > To: d...@lucene.apache.org; gmcdon...@apache.org; 'Uwe Schindler' > > <uschind...@apache.org> > > > Cc: 'builds' <builds@apache.org> > > > Subject: RE: JDK, Maven, Ant versions have been consolidated. > > > > > > Hi, > > > > > > the test files could be a puppet module. > > > > > > The lucene.build.properties in the Jenkins home dir is a “hardware > specific” > > file, as it allows to configure the number of parallel runners and where > the > > special test files are – one would not be able to use it on any other > jenkins > > slave. If it is missing, defaults are used, which are not good for > optimal CPU > > use. We could add them also to the puppet module, but this would make it > > harder for use to change the file easily. > > > > > > All other files are unmodified. > > > > > > Uwe > > > > > > ----- > > > Uwe Schindler > > > uschind...@apache.org > > > ASF Member, Apache Lucene PMC / Committer > > > Bremen, Germany > > > http://lucene.apache.org/ > > > > > > From: Gav [mailto:gmcdon...@apache.org] > > > Sent: Monday, August 15, 2016 5:36 AM > > > To: Uwe Schindler <uschind...@apache.org> > > > Cc: builds <builds@apache.org>; d...@lucene.apache.org > > > Subject: Re: JDK, Maven, Ant versions have been consolidated. > > > > > > I'll make a copy of the jenkins home dir and get started. > > > It might be an idea to add those test files to puppet wdyt? > > > (I see nothing private about them) > > > > > > Gav... > > > > > > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschind...@apache.org> > > wrote: > > >> Hi Gav, > > >> > > >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this > spammed > > the mailing list with like 50 mails today. Can you send me a msg when you > > installed the required puppet modules? > > >> > > >> Uwe > > >> > > >> ----- > > >> Uwe Schindler > > >> uschind...@apache.org > > >> ASF Member, Apache Lucene PMC / Committer > > >> Bremen, Germany > > >> http://lucene.apache.org/ > > >> > > >> From: Gav [mailto:gmcdon...@apache.org] > > >> Sent: Friday, August 12, 2016 11:18 AM > > >> To: Uwe Schindler <uschind...@apache.org> > > >> Cc: builds <builds@apache.org> > > >> Subject: Re: JDK, Maven, Ant versions have been consolidated. > > >> > > >> Hi Uwe, > > >> > > >> Will get on with your vm tomorrow. > > >> > > >> Gav... > > >> > > >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschind...@apache.org > > > > wrote: > > >>> Hi, > > >>> > > >>> on the lucene-jenkins machine, the packages are still missing. I > think you > > have to install the jenkins puppet module. Here the message: > > >>> FATAL: Cannot find executable from the chosen Ant installation "Ant > > 1.8.4" > > >>> > > >>> See my e-mail from last week about using the "official" jenkins > puppet > > module (answer to your question). Unless our customizations in the home > > dir keep preserved, we are fine with that: > > >>> > > >>> My mail: > > >>> > > >>> ==================================== > > >>> Hi Gav, > > >>> > > >>> sorry for responding late – I am currently on vacation! > > >>> > > >>> About your questions: In the past we required some extra modules > (like > > older Subversion to be compatible with the older Jenkins Checkouts). As > we > > are on Git now and the build scripts use JGit to fetch revision numbers > and > > check for clean checkouts, this is obsolete. So if you want to install > the > > “Jenkins puppet module”, we are fine. > > >>> > > >>> There is one special case for Lucene: > > >>> We have a hardware-specific “lucene.build.properties” file in > Jenkin’s > > root and also some additional “test files” (large wikiepdia checkouts) > that are > > used for nightly tests. If the puppet module does not delete those files > in > > current ~/jenkins, it is ok. We would still need access to the machine > and > > sudo to jenkins. > > >>> > > >>> If this is all fine, go ahead! > > >>> > > >>> Uwe > > >>> ======================================= > > >>> > > >>> Uwe > > >>> > > >>> ----- > > >>> Uwe Schindler > > >>> H.-H.-Meier-Allee 63, D-28213 Bremen > > >>> http://www.thetaphi.de > > >>> eMail: u...@thetaphi.de > > >>> > > >>> > -----Original Message----- > > >>> > From: Gav [mailto:gmcdon...@apache.org] > > >>> > Sent: Friday, August 12, 2016 6:43 AM > > >>> > To: builds <builds@apache.org> > > >>> > Subject: JDK, Maven, Ant versions have been consolidated. > > >>> > > > >>> > HI All, > > >>> > > > >>> > As per previous emails and as documented [1] [2] [3] we have now > > >>> > consolidated the JDK, Maven and Ant install versions. > > >>> > > > >>> > All versions are now installed/managed via puppet and any 'auto- > > install' > > >>> > versions have been removed. > > >>> > > > >>> > At the time of consolidation and removal of versions, over 200 jobs > > were > > >>> > still configured to use deprecated versions despite the emails > I've been > > >>> > sending. Those 200+ jobs I have migrated myself to supported > > versions. > > >>> > > > >>> > Any issues with your jobs please do create a new INFRA jira or > email > > this > > >>> > list. > > >>> > > > >>> > Thanks, > > >>> > > > >>> > Gav... > > >>> > > > >>> > [1] - > > >>> > > > https://cwiki.apache.org/confluence/display/INFRA/JDK+ > Installation+Matrix > > >>> > [2] - > > >>> > > > https://cwiki.apache.org/confluence/display/INFRA/Ant+ > Installation+Matrix > > >>> > [3] - > > >>> > > > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat > > >>> > rix > > >>> > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > >
