Who is creating all these dragon JIRAs?
Colin
+1
Thanks, Andrew. This will avoid so many spurious conflicts when
cherry-picking changes, and so much wasted time on commit.
best,
Colin
On Thu, Mar 3, 2016 at 9:11 PM, Andrew Wang wrote:
> Hi all,
>
> With the inclusion of HADOOP-12651 going back to branch-2.8, CHANGES.txt
> and release note
e released in 2.9. I think we
> should rather concentrate our EC dev efforts to harden key features under
> the follow-on umbrella HDFS-8031 and make it solid for a 3.0 release.
>
> Sincerely,
> Zhe
>
> On Mon, Feb 22, 2016 at 9:25 AM Colin P. McCabe wrote:
>
>&g
+1 for a release of 3.0. There are a lot of significant,
compatibility-breaking, but necessary changes in this release... we've
touched on some of them in this thread.
+1 for a parallel release of 2.8 as well. I think we are pretty close
to this, barring a dozen or so blockers.
best,
Colin
On
It's great to see interest in improving this functionality. I think
Chimera could be successful as an Apache project. I don't have a
strong opinion one way or the other as to whether it belongs as part
of Hadoop or separate.
I do think there will be some challenges splitting this functionality
o
On Mon, Nov 23, 2015 at 1:53 PM, Colin P. McCabe wrote:
> I agree that our tests are in a bad state. It would help if we could
> maintain a list of "flaky tests" somewhere in git and have Yetus
> consider the flakiness of a test before -1ing a patch. Right now, we
> pre
I agree that our tests are in a bad state. It would help if we could
maintain a list of "flaky tests" somewhere in git and have Yetus
consider the flakiness of a test before -1ing a patch. Right now, we
pretty much all have that list in our heads, and we're not applying it
very consistently. Hav
Looks like a good idea. I assume you are targetting this only at trunk /
3.0 based on the "target version" and the incompatibility discussion?
best,
Colin
On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa wrote:
> Hi Steve,
>
> Thanks for your help.
>
> > 2. it's "significant"
>
> This change in
Thanks for being proactive here, Steve. I think this is a good example of
why this change should have been done in a branch rather than having been
done directly in trunk.
regards,
Colin
On Wed, Oct 14, 2015 at 10:36 AM, Steve Loughran
wrote:
> just an FYI, the split off of hadoop hdfs into c
I think it makes sense to have a 2.8 release since there are a
tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
release seems like something we should consider separately since it
would not have the same compatibility guarantees as a 2.8 release.
There's a pretty big delta betwee
Hi Mohammad,
Like ATM said, HDFS-8965 is an important fix in this area. We have
found that it prevents cases where INotify tries to read invalid
sequences of bytes (sometimes because the edit log was truncated or
corrupted; other times because it is in the middle of a write).
HDFS-8964 fixes the
Has anyone measured the overhead of running SASL on
DataTransferProtocol? I would expect it to be non-zero compared with
simply running on a low port. The CPU overhead especially could
regress performance on a typical Hadoop cluster.
best,
Colin
On Thu, Sep 10, 2015 at 9:55 AM, Chris Nauroth w
On Mon, Jun 1, 2015 at 3:21 AM, Steve Loughran wrote:
>
> HADOOP-12009 (https://issues.apache.org/jira/browse/HADOOP-12009) patches the
> FS javadoc and contract tests to say "the order you get things back from a
> listStatus() isn't guaranteed to be alphanumerically sorted"
>
> That's one of th
On Thu, Jun 4, 2015 at 2:46 PM, Rahul Shrivastava wrote:
> Hi,
>
>
> Suppose I write a Java client to create a directory on HDFS. Is there a way
> to tag this request and get the tagged information on NameNode via
> DFSInotifyEventInputStream or otherwise ?
>
> In short, is there a way to give opt
Thanks, Allen. This has long been a thorn in our side, and it's
really good to see someone work on it.
cheers,
Colin
On Tue, May 5, 2015 at 2:59 PM, Allen Wittenauer wrote:
> TL;DR:
>
> Heads up: I’m going to hack on these scripts to fix the race
> conditions.
>
>
>
> Pre
Sorry for the late reply. It seems like the consensus is that we
should push these fixes to 2.7.1. That works for me. HADOOP-11802
should be in there soon, hopefully the rest will follow quickly.
best,
Colin
On Wed, Apr 22, 2015 at 4:27 PM, Vinod Kumar Vavilapalli
wrote:
> It took a while for
Thanks, Andrew and Joep.
+1 for maintaining wire and API compatibility, but moving to JDK8 in 3.0
best,
Colin
On Mon, Mar 16, 2015 at 3:22 PM, Andrew Wang wrote:
> I took the liberty of adding line breaks to Joep's mail.
>
> Thanks for the great feedback Joep. The goal with 3.x is to maintain A
> changes
>>>>> > >> in these test suites is HDFS-7722. That patch still looks fine
>>>>> > though. I
>>>>> > >> don¹t know if there are other uncommitted patches that changed these
>>>>> > test
>>>>> > >&
Hi all,
A very quick (and not thorough) survey shows that I can't find any
jenkins jobs that succeeded from the last 24 hours. Most of them seem
to be failing with some variant of this message:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-clean-plugin:2.5:clean (default-clean)
o
way for the current
> plan on hadoop-3.x right? So, I don't see the difference?
>
> Arun
>
>
> From: Colin P. McCabe
> Sent: Monday, March 09, 2015 3:05 PM
> To: hdfs-dev@hadoop.apache.org
> Cc: mapreduce-...@hadoop.apache
Er, that should read "as Allen commented" C.
On Tue, Mar 10, 2015 at 11:55 AM, Colin P. McCabe wrote:
> Hi Arun,
>
> Not all changes which are incompatible can be "fixed"-- sometimes an
> incompatibility is a necessary part of a change. For example, taking
&
Java 7 will be end-of-lifed in April 2015. I think it would be unwise
to plan a new Hadoop release against a version of Java that is almost
obsolete and (soon) no longer receiving security updates. I think
people will be willing to roll out a new version of Java for Hadoop
3.x.
Similarly, the wh
I agree with Andrew and Konst here. I don't think the language is
unclear in the rule, either... "consensus with a minimum of one +1"
clearly indicates that _other people_ are involved, not just one
person.
I would also mention that we created the "branch committer" role
specifically to make it e
Thanks for bringing this up. If you can find any place where an array
might realistically be larger than 67 million elements, then I guess
file a JIRA for it. Also this array needs to be of objects, not of
primitives (quicksort is used for those in jdk7, apparently). I can't
think of any such pl
eed to download dependencies
> fresh every time.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
>
>
>
> On 2/12/15, 2:00 PM, "Colin P. McCabe" wrote:
>
>>We could potentially use different .m2 directories for each executor.
>>I t
In the past, "block size" and "size of block N" were completely
separate concepts in HDFS.
The former was often referred to as "default block size" or "preferred
block" size or some such thing. Basically it was the point at which
we'd call it a day and move on to the next block, whenever any bloc
In general, the DN does not perform reads from files under a big lock.
We only need the lock for protecting the replica map and some of the
block state. This lock hasn't really been a big problem in the past
and I would hesitate to add complexity here (although I haven't
thought about it that hard
Hi Demai,
Nearly all input and output stream operations will talk directly to
the DN without involving the NN. The NameNode is involved in metadata
operations such as renaming or opening files, not in reading data.
Hope this helps.
best,
Colin
On Thu, Feb 12, 2015 at 4:21 PM, Demai Ni wrote:
of our missing class error issues.
Colin
On Tue, Feb 10, 2015 at 2:13 AM, Steve Loughran wrote:
> Mvn is a dark mystery to us all. I wouldn't trust it not pick up things from
> other builds if they ended up published to ~/.m2/repository during the process
>
>
>
> On 9 Febr
I'm sorry, I don't have any insight into this. With regard to
HADOOP-11084, I thought that $BUILD_URL would be unique for each
concurrent build, which would prevent build artifacts from getting
mixed up between jobs. Based on the value of PATCHPROCESS that Kihwal
posted, perhaps this is not the c
Hi Niels,
I agree that direct-attached storage seems more economical for many users.
As an HDFS developer, I certainly have a dog in this fight as well :)
But we should be respectful towards people trying to contribute code to
Hadoop and evaluate the code on its own merits. It is up to our users
Hi dlmarion,
In general, any upgrade process we do will consume disk space, because
it's creating hardlinks and a new "current" directory, and so forth.
So upgrading when disk space is very low is a bad idea in any
scenario. It's certainly a good idea to free up some space before
doing the upgrad
32 matches
Mail list logo
