pls remove me from this group
On Tue, Sep 22, 2015 at 8:26 PM, Steve Loughran
wrote:
>
> > On 22 Sep 2015, at 12:16, Brahma Reddy Battula <
> brahmareddy.batt...@huawei.com> wrote:
> >
> > After using timestamped jars, hadoop-hdfs module might still continue to
> use earlier timestamped jars (co
yetus-5 was just committed which does all of this (and more, of course).
On Oct 6, 2015, at 2:35 AM, Steve Loughran wrote:
>
>> On 5 Oct 2015, at 19:45, Colin McCabe wrote:
>>
>> On Mon, Sep 28, 2015 at 12:52 AM, Steve Loughran
>> wrote:
>>>
>>> the jenkins machines are shared across mult
> On 5 Oct 2015, at 19:45, Colin McCabe wrote:
>
> On Mon, Sep 28, 2015 at 12:52 AM, Steve Loughran
> wrote:
>>
>> the jenkins machines are shared across multiple projects; cut the executors
>> to 1/node and then everyone's performance drops, including the time to
>> complete of all jenkins
On Mon, Sep 28, 2015 at 12:52 AM, Steve Loughran wrote:
>
> the jenkins machines are shared across multiple projects; cut the executors
> to 1/node and then everyone's performance drops, including the time to
> complete of all jenkins patches, which is one of the goals.
Hi Steve,
Just to be cl
> On 28 Sep 2015, at 10:05, Vinayakumar B wrote:
>
> Setting the version to unique value sounds reasonable.
>
> Is there anyway in mvn to clean such artifacts installed.. as part of
> cleanup in the same build instead of nightly cleanup?
>
Well, there's a dependency:purge-local-repository ma
Setting the version to unique value sounds reasonable.
Is there anyway in mvn to clean such artifacts installed.. as part of
cleanup in the same build instead of nightly cleanup?
-Vinay
On Sep 28, 2015 1:22 PM, "Steve Loughran" wrote:
>
> the jenkins machines are shared across multiple projects
the jenkins machines are shared across multiple projects; cut the executors to
1/node and then everyone's performance drops, including the time to complete of
all jenkins patches, which is one of the goals.
https://builds.apache.org/computer/
Like I said before: I don't think we need one mvn r
I think the right route is to file an INFRA JIRA with this request. Not
entirely sure since at one point the Hadoop build infra was separately
managed by Yahoo, but I think as of late it's under Apache administration.
Best,
Andrew
On Fri, Sep 25, 2015 at 9:43 PM, Vinayakumar B
wrote:
> Thanks A
Thanks Andrew,
May be we can try making it to 1 exec, and try for sometime. i think also
need to check what other jobs, hadoop ecosystem jobs, run in Hadoop nodes.
As HADOOP-11984 and HDFS-9139 are on the way to reduce build time
dramatically by enabling parallel tests, HDFS and COMMON precommit b
Thanks for checking Vinay. As a temporary workaround, could we reduce the #
of execs per node to 1? Our build queues are pretty short right now, so I
don't think it would be too bad.
Best,
Andrew
On Wed, Sep 23, 2015 at 12:18 PM, Vinayakumar B
wrote:
> In case if we are going to have separate r
In case if we are going to have separate repo for each executor,
I have checked, each jenkins node is allocated 2 executors. so we only need
to create one more replica.
Regards,
Vinay
On Wed, Sep 23, 2015 at 7:33 PM, Steve Loughran
wrote:
>
> > On 22 Sep 2015, at 16:39, Colin P. McCabe wrote:
> On 22 Sep 2015, at 16:39, Colin P. McCabe wrote:
>
>> ANNOUNCEMENT: new patches which contain hard-coded ports in test runs will
>> henceforth be reverted. Jenkins matters more than the 30s of your time it
>> takes to use the free port finder methods. Same for any hard code paths in
>> file
There are multiple problems with just spamming test patch with local repos.
I've done a not insignificant amount of investigation in this space and there
reasons why I didn't just slam it in even though I've been aware of the issue
for a very long time. There are specific reasons why I want to
>
>
> Did anyone address Andrew's proposal to have one private repo per
> Jenkins executor? That seems like the simplest approach to me. It
> seems like that would only generate more network traffic in the case
> where a dependency changes, which should be relatively rare.
>
> We're blocked on YE
On Tue, Sep 22, 2015 at 9:09 PM, Colin P. McCabe wrote:
> On Mon, Sep 21, 2015 at 4:08 AM, Steve Loughran
> wrote:
> >
> >> On 19 Sep 2015, at 04:42, Allen Wittenauer wrote:
> >>
> >> a) Multi-module patches are always troublesome because it makes the
> test system do significantly more work.
On Mon, Sep 21, 2015 at 4:08 AM, Steve Loughran wrote:
>
>> On 19 Sep 2015, at 04:42, Allen Wittenauer wrote:
>>
>> a) Multi-module patches are always troublesome because it makes the test
>> system do significantly more work. For Yetus, we've pared it down as far as
>> we can go to get *some*
> On 22 Sep 2015, at 12:16, Brahma Reddy Battula
> wrote:
>
> After using timestamped jars, hadoop-hdfs module might still continue to use
> earlier timestamped jars (correct) and may complete run.But later modules
> might refer to updated jars which are from some other build.
why?
If I d
esystems.
Good Idea.
Thanks & Regards
Brahma Reddy Battula
From: Steve Loughran [ste...@hortonworks.com]
Sent: Monday, September 21, 2015 4:38 PM
To: common-dev@hadoop.apache.org
Subject: Re: Local repo sharing for maven builds
> On 19 Sep 2015,
> On 19 Sep 2015, at 04:42, Allen Wittenauer wrote:
>
> a) Multi-module patches are always troublesome because it makes the test
> system do significantly more work. For Yetus, we've pared it down as far as
> we can go to get *some* speed increases, but if a patch does something like
> hit e
Andrew Wang wrote:
Theoretically, we should be able to run unittests without a full `mvn
install` right? The "test" phase comes before "package" or "install", so I
figured it only needed class files. Maybe the multi-module-ness screws this
up.
Unless something weird is configured in the poms (w
I just filed YETUS-4 for supporting additional maven args.
https://issues.apache.org/jira/browse/YETUS-4
Theoretically, we should be able to run unittests without a full `mvn
install` right? The "test" phase comes before "package" or "install", so I
figured it only needed class files. Maybe the m
On Fri, Sep 18, 2015 at 2:42 PM, Allen Wittenauer wrote:
> As far as Yetus goes, we've got a JIRA open to provide for per-instance
> caches when
> using the docker container code. I've got it in my head how I think we can do
> it, but just
> haven't had a chance to code it. So once that gets wr
I
>> don't
>>>> think we can just set it before calling test-patch, since it'd get
>>> squashed
>>>> by setup_defaults.
>>>>
>>>> Allen/Chris/Yetus folks, any guidance here?
>>>>
>>>> Thanks,
>>>
> Allen/Chris/Yetus folks, any guidance here?
> > >
> > > Thanks,
> > > Andrew
> > >
> > > On Fri, Sep 18, 2015 at 11:55 AM, wrote:
> > >
> > >> You can use one per build processor, that reduces concurrent updates
> but
> > >&g
2015 at 11:55 AM, wrote:
> >
> >> You can use one per build processor, that reduces concurrent updates but
> >> still keeps the cache function. And then try to avoid using install.
> >>
> >> --
> >> http://bernd.eckenfels.net
> >>
> >>
keeps the cache function. And then try to avoid using install.
>>
>> --
>> http://bernd.eckenfels.net
>>
>> -Original Message-
>> From: Andrew Wang
>> To: "common-dev@hadoop.apache.org"
>> Cc: Andrew Bayer , Sangjin Lee ,
>> Lei Xu , infrastr
, infrastruct...@apache.org
> Sent: Fr., 18 Sep. 2015 20:42
> Subject: Re: Local repo sharing for maven builds
>
> I think each job should use a maven.repo.local within its workspace like
> abayer said. This means lots of downloading, but it's isolated.
>
> If we care ab
, Lei
Xu , infrastruct...@apache.org
Sent: Fr., 18 Sep. 2015 20:42
Subject: Re: Local repo sharing for maven builds
I think each job should use a maven.repo.local within its workspace like
abayer said. This means lots of downloading, but it's isolated.
If we care about download time
Are we using maven.repo.local in our pre-commit or commit jobs? We cannot
see the configuration of these jenkins jobs.
On Fri, Sep 18, 2015 at 11:41 AM, Andrew Wang
wrote:
> I think each job should use a maven.repo.local within its workspace like
> abayer said. This means lots of downloading, bu
I think each job should use a maven.repo.local within its workspace like
abayer said. This means lots of downloading, but it's isolated.
If we care about download time, we could also bootstrap with a tarred
.m2/repository after we've run a `mvn compile`, so before it installs the
hadoop artifacts.
+hadoop common dev. Any suggestions?
On Fri, Sep 18, 2015 at 10:41 AM, Andrew Bayer
wrote:
> You can change your maven call to use a different repository - I believe
> you do that with -Dmaven.repository.local=path/to/repo
> On Sep 18, 2015 19:39, "Ming Ma" wrote:
>
>> Hi,
>>
>> We are seeing
31 matches
Mail list logo
