We also issue explicit readahead via fadvise since 2011 or so, so the
typical io sizes hitting the device are large enough to max out the
throughput, at least for typical spinning disks.
Todd
On Wed, Mar 11, 2020, 9:46 AM Kihwal Lee
wrote:
> When Datanode was initially designed, Linux AIO was s
in Hadoop. But, with some of the changes in the proposed 2.0.3-alpha, it
wouldn't be wire-protocol-stable.
How can we best devise a scheme that explains the various factors above in
a more detailed way than one big red warning sticker? What of the above
factors does the community think would be implied by "GA?"
Thanks
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
/browse/HDFS-347?focusedCommentId=13551755&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13551755
>
--
Todd Lipcon
Software Engineer, Cloudera
gt; -1
> The patch seems not ready yet. I have posted some comments/suggestions on
> the JIRA. Colin also has agreed that there are some bugs to be fixed. Sorry.
>
> Tsz-Wo
>
>
>
>
> ____
> From: Todd Lipcon
> To: hdfs-dev@hadoop.ap
currently patch is still not yet ready. It seems to have unnecessarily
> changed the API and protocol. I believe those are important but not trivial
> things.
>
> Tsz-Wo
>
>
>
> From: Todd Lipcon
> To: hdfs-dev@hadoop.apache.org; Ts
active
review" to me. On the other hand, I spent probably 40-50% of my time
over the last month reviewing and testing this branch and have voted
+1.
-Todd
>
>
> On Wed, Feb 20, 2013 at 12:16 PM, Todd Lipcon wrote:
>
>> Hi Nicholas,
>>
>> I looked at your comme
ies are owned by an HDFS
user, and users of the cluster run under other unix credentials.
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
On Wed, Feb 20, 2013 at 3:06 PM, Todd Lipcon wrote:
> https://issues.apache.org/jira/browse/HDFS-4476 where I pointed out
sorry, meant to link to:
https://issues.apache.org/jira/browse/HDFS-2246?focusedCommentId=13102013&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-t
optimization on a platform which is not
yet officially supported on trunk, especially when the old code path
is _insecure_ and thus unusable in most environments.
Todd
>
> ____
> From: Todd Lipcon
> To: hdfs-dev@hadoop.apache.org; Tsz Wo Sze
> Sent: Wednesd
small number of developers who know how to program
against that platform.
If that's the axe hanging over our head with the Windows branch, then
I'm all for saying "good, keep it on a branch and don't merge it to
trunk".
I was hoping we could all work together a bit be
ckReaderLocal: simpler implementation that uses raw FileChannel
objects. We don't need to cache anything, or make RPCs to the
DataNode."
from his 10/1/2012 patch upload. So, any patch you might have looked
at since then would have clearly removed the old code path.
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
;
> [1]
> https://issues.apache.org/jira/browse/HDFS-347?focusedCommentId=13579704&p
> age=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comme
> nt-13579704
>
> [2]
> https://issues.apache.org/jira/browse/HDFS-347?focusedCommentId=13551755&p
> age=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comme
> nt-13551755
>
--
Todd Lipcon
Software Engineer, Cloudera
; >>> The support for Hadoop on Windows was proposed in HADOOP-8079<
> > >>> https://issues.apache.org/jira/browse/HADOOP-8079> almost a year
> ago.
> > >>>The
> > >>> goal was to make Hadoop natively integrated, full-featured, and
> > >>>performance
> > >>> and scalability tuned on Windows Server or Windows Azure.
> > >>> We are happy to announce that a lot of progress has been made in this
> > >>> regard.
> > >>>
> > >>> Initial work started in a feature branch, branch-1-win, based on
> > >>>branch-1.
> > >>> The details related to the work done in the branch can be seen in
> > >>> CHANGES.txt<
> > >>>
> > >>>
> > http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/CHANGES
> .
> > >>>branch-1-win.txt?view=markup
> > >>> >.
> > >>> This work has been ported to a branch, branch-trunk-win, based on
> > trunk.
> > >>> Merge patch for this is available on
> > >>> HADOOP-8562<https://issues.apache.org/jira/browse/HADOOP-8562>
> > >>> .
> > >>>
> > >>> Highlights of the work done so far:
> > >>> 1. Necessary changes in Hadoop to run natively on Windows. These
> > changes
> > >>> handle differences in platforms related to path names, process/task
> > >>> management etc.
> > >>> 2. Addition of winutils tools for managing file permissions and
> > >>>ownership,
> > >>> user group mapping, hardlinks, symbolic links, chmod, disk
> utilization,
> > >>>and
> > >>> process/task management.
> > >>> 3. Added cmd scripts equivalent to existing shell scripts
> > >>> hadoop-daemon.sh, start and stop scripts.
> > >>> 4. Addition of block placement policy implemnation to support cloud
> > >>> enviroment, more specifically Azure.
> > >>>
> > >>> We are very close to wrapping up the work in branch-trunk-win and
> > >>>getting
> > >>> ready for a merge. Currently the merge patch is passing close to 100%
> > of
> > >>> unit tests on Linux. Soon I will call for a vote to merge this branch
> > >>>into
> > >>> trunk.
> > >>>
> > >>> Next steps:
> > >>> 1. Call for vote to merge branch-trunk-win to trunk, when the work
> > >>> completes and precommit build is clean.
> > >>> 2. Start a discussion on adding Jenkins precommit builds on windows
> and
> > >>> how to integrate that with the existing commit process.
> > >>>
> > >>> Let me know if you have any questions.
> > >>>
> > >>> Regards,
> > >>> Suresh
> > >>>
> > >>>
> > >>
> > >>
> > >>--
> > >>http://hortonworks.com/download/
> > >
> >
>
>
>
> --
> http://hortonworks.com/download/
>
--
Todd Lipcon
Software Engineer, Cloudera
; I have only three nodes(A,B,C) for test
> A and B are name nodes.
> B and C are data nodes.
> ZK deployed on A, B and C.
>
> A, B and C are all journal nodes.
>
> Thanks.
>
--
Todd Lipcon
Software Engineer, Cloudera
as been addressed-- see HDFS-4538 for details.
> Suresh has also volunteered to maintain the insecure SCR code until secure
> SCR can be implemented for Windows.
>
> Please cast your vote by EOD Monday 4/8.
>
> best,
> Colin
>
--
Todd Lipcon
Software Engineer, Cloudera
Suresh has also volunteered to maintain the insecure SCR code until secure
> SCR can be implemented for Windows.
>
> Please cast your vote by EOD Monday 4/8.
>
> best,
> Colin
>
--
Todd Lipcon
Software Engineer, Cloudera
fic APIs
rather than any standard Linux interface. So, I'm in favor of either
putting the APIs in HdfsAdmin, or alternatively in DistributedFileSystem,
forcing a user to down-cast if they want to use the HDFS-specific operation.
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
} while (thisListing.hasMore());
>
> return listing.toArray(new FileStatus[listing.size()]);
> }
>
> --------
>
>
>
>
>
> Ref:
>
> https://svn.apache.org/repos/asf/hadoop/common/tags/release-1.0.4/src/hdfs/org/apache/hadoop/hdfs/DistributedFileSystem.java
> http://docs.oracle.com/javase/6/docs/api/java/util/Iterator.html
>
>
> thanks!
>
> -bc
>
--
Todd Lipcon
Software Engineer, Cloudera
en the keepalive, or shorten the socket cache expiry, or both.
>
> thoughts?
> Colin
>
--
Todd Lipcon
Software Engineer, Cloudera
of TestCLI against a fully distributed
> cluster. At this
> point I'm actually reasonably concerned and will follow up tomorrow.
>
> Thanks,
> Roman.
>
--
Todd Lipcon
Software Engineer, Cloudera
re a project lead for one of the Apache projects, look at the
> schedule below and suggest a few slots when you think it would be best for
> your project to meet. I'll try to work out a schedule where no more than 2
> projects are using the lounge at once.
> >
> > Day 1, 26th June: 10:30am - 12:30pm, 1:45pm - 3:30pm, 3:45pm - 5:00pm
> >
> > Day 2, 27th June: 10:30am - 12:30pm, 1:45pm - 3:30pm, 3:45pm - 5:00pm
> >
> > It will be up to you, the hadoop contributors, from there.
> >
> > Look forward to seeing you all at the summit,
> >
> > E14
> >
> > PS Please forward to the other -dev lists. This event is for folks on
> the -dev lists.
> >
>
>
>
>
> --
> http://hortonworks.com/download/
>
--
Todd Lipcon
Software Engineer, Cloudera
w what your experience with this format is.
>
> E14
>
> On Jun 24, 2013, at 2:50 PM, Todd Lipcon wrote:
>
> > Hey all,
> >
> > Unfortunately I will have to leave early on Thursday, so will miss the
> slotted time at the design lounge.
> >
> > But, I'l
M, Azuryy Yu wrote:
> > Hi dear All,
> >
> > It's long type for the txid currently,
> >
> > FSImage.java:
> >
> > boolean loadFSImage(FSNamesystem target, MetaRecoveryContext recovery)
> > throws IOException{
> >
> > editLog.setNextTxId(lastAppliedTxId + 1L);
> > }
> >
> > Is it possible that (lastAppliedTxId + 1L) exceed Long.MAX_VALUE ?
>
>
>
> --
> Harsh J
>
--
Todd Lipcon
Software Engineer, Cloudera
t; is
> > 19th July. So, I'm pretty tensed without any mentors :(
> >
> > [1] https://issues.apache.org/jira/browse/ACCUMULO-1197
> > [2] https://issues.apache.org/jira/browse/HDFS-4680
> > [3] https://github.com/cloudera/htrace
> > [4] http://community.apache.org/mentoringprogramme-icfoss-pilot.html
> > [5] https://issues.apache.org/jira/browse/HBASE-6449
> >
> > Thank you,
> > Sreejith R
> >
>
--
Todd Lipcon
Software Engineer, Cloudera
cdh/4/hadoop-2.0.0-cdh4.**1.1.tar.gz<http://archive.cloudera.com/cdh4/cdh/4/hadoop-2.0.0-cdh4.1.1.tar.gz>
>
> We would greatly appreciate any advise on whether or not this approach
> sounds reasonable, and if you would consider accepting these modifications
> into the official Hadoop code base.
>
> Thank you,
> Jeff, Alja & Matevz
> UCSD Physics
>
--
Todd Lipcon
Software Engineer, Cloudera
House
> > Jacklyns Lane
> > Alresford
> > SO24 9JJ
> > Tel: +44 (0)1962 738232
> > Mob: +44 (0)7710 160458
> > Fax: +44 (0)1962 600112
> > Web: www.b <http://www.fosolutions.co.uk/>espokesoftware.com<
> http://bespokesoftware.com/>
> >
ormation seems regular. The problem is that between the 2 lines
> there's no log in 12 minutes. There is no long gc happened. It seems the
> code blocked somewhere. Unfortunately, I forgot to print the jstack info
> T_T.
>
> Hope for your response.
>
> Best regards,
> Mickey
>
--
Todd Lipcon
Software Engineer, Cloudera
; as part of persistent state. I
> want to know whether the case 3 is handled in CDH4.3.1 version. If the
> case is hadnled, where is the code?
>
>
> Thanks,
>
> LiuLe
>
--
Todd Lipcon
Software Engineer, Cloudera
just trying to find out if it is indeed
abandoned)
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
> >> >> --
> >> >> http://hortonworks.com/download/
> >> >>
> >> >> --
> >> >> CONFIDENTIALITY NOTICE
> >> >> NOTICE: This message is intended for the use of the individual or
> >> >>entity to
> >> >> which it is addressed and may contain information that is
> >>confidential,
> >> >> privileged and exempt from disclosure under applicable law. If the
> >> >>reader
> >> >> of this message is not the intended recipient, you are hereby
> >>notified
> >> >>that
> >> >> any printing, copying, dissemination, distribution, disclosure or
> >> >> forwarding of this communication is strictly prohibited. If you have
> >> >> received this communication in error, please contact the sender
> >> >>immediately
> >> >> and delete it from your system. Thank You.
> >> >>
> >> >
> >> >
> >> >
> >> >--
> >> >Have a Nice Day!
> >> >Lohit
> >>
> >>
> >
> >
> >--
> >http://hortonworks.com/download/
> >
> >--
> >CONFIDENTIALITY NOTICE
> >NOTICE: This message is intended for the use of the individual or entity
> >to
> >which it is addressed and may contain information that is confidential,
> >privileged and exempt from disclosure under applicable law. If the reader
> >of this message is not the intended recipient, you are hereby notified
> >that
> >any printing, copying, dissemination, distribution, disclosure or
> >forwarding of this communication is strictly prohibited. If you have
> >received this communication in error, please contact the sender
> >immediately
> >and delete it from your system. Thank You.
>
>
--
Todd Lipcon
Software Engineer, Cloudera
On Mon, Nov 11, 2013 at 2:57 PM, Colin McCabe wrote:
> To be honest, I'm not aware of anything in 2.2.1 that shouldn't be
> there. However, I have only been following the HDFS and common side
> of things so I may not have the full picture. Arun, can you give a
> specific example of something you
in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
Todd Lipcon
Software Engineer, Cloudera
no public APIs. The existing tests include both
> unit tests and end-to-end tests. They have already covered this
> feature.
>
> Once the feature is merged into trunk, we will continue to test and
> fix any bugs that may be found on trunk.
>
> The bulk of the design and imp
o bring up kerberos to run it like that. And for more
> sophisticated
> > multinode deployments, should the scripts refuse to work without
> kerberos
> > unless you pass in some argument like "--Dinsecure-clusters-
> permitted"
> >
> > Any other ideas?
> >
> >
> >
> -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
> >
>
>
>
--
Todd Lipcon
Software Engineer, Cloudera
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2150/
---
Review request for hadoop-hdfs and Todd Lipcon.
Summary
---
Uploading
623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
<https://reviews.apache.org/r/2150/#comment5251>
this should at least be warn, right? and passing e as the second argument
is better since
into some kind of attic/ directory
Thoughts?
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
> On 2011-10-03 18:58:13, Todd Lipcon wrote:
> > just a few nits, mostly looks good. A few questions I have that aren't
> > directly related to this patch:
> > - is SafeMode now a replicated thing, or does each NN separately enter
> > safemode? I thi
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2150/
---
(Updated 2011-10-06 23:25:18.391776)
Review request for hadoop-hdfs and Todd
r HA NN), right?
- Todd
On 2011-10-06 23:25:18, Todd Lipcon wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2150/
> ---
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2672/#review3014
---
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/
; BlocksBeingWritten directory. (Also the previous generation timestamp)
>
> Looks very strange for me.
> Does this means that the finalized block file & meta file (which is written
> in current folder) is getting lost after DN restart
> Due to which Namenode will not rece
em.addStoredBlock: addStoredBlock request received for
> blk_1321803251510_83627 on xx.xx.132.22:10010 size 104428544 but was
> rejected: Reported as block being written but is a block of closed file.
>
> Regards,
> Uma
>
>
> From: Todd
ncludes HDFS-1539, you could consider
setting dfs.datanode.synconclose to true, which would have prevented
this problem.
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
r trigger: Failure
###
## FAILED TESTS (if any)
##
No tests ran.
--
Todd Lipcon
Software Engineer, Cloudera
at
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:81)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
--
Todd Lipcon
Software Engineer, Cloudera
ntrib
On Mon, Dec 19, 2011 at 10:13 AM, Todd Lipcon wrote:
> HDFS Builds have been failing since HDFS-234 (bk support) went in 6
> days ago. Can someone please fix this? Otherwise we should revert the
> patch.
>
> -Todd
>
>
> -- Forwarded message --
I committed this. (r1220940)
-Todd
On Mon, Dec 19, 2011 at 11:49 AM, Todd Lipcon wrote:
> I'm going to commit the following fix if no one screams otherwise:
>
> Index: hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkj
Konstantin - I think the BN-based approach is probably susceptible to
the same issue, so curious if you've solved it in your implementation.
Thanks
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
updateBlocks" call inside FSEditLogLoader. I used that existing
implementation rather than adding the new one in FSDirectory, since
this function had some other changes related to HA in the branch
version.
I'll wait for a +1 before committing. I ran all of the unit tests and
they passed.
-Tod
27;t go with my suggestion in (2), please add a
> comment explaining the assignment.
>
> Otherwise looks good. Merge away.
>
> --
> Aaron T. Myers
> Software Engineer, Cloudera
>
>
>
> On Fri, Feb 3, 2012 at 2:10 PM, Todd Lipcon wrote:
>
>> I've got a merg
luster performance
> against the branch. Would be good to have numbers for both cases with
> HA off and HA on.
>
> I'll post these questions to the jira as well.
>
> Thanks,
> --Konstantin
>
> On Wed, Feb 8, 2012 at 9:08 PM, Todd Lipcon wrote:
>> The branch develo
on to 0.17 or 0.18 (which seems entirely reasonable given
they're ~4 years old).
Will report back when a new patch is up for review.
-Todd
>
> Thanks,
> --Konstantin
>
> On Thu, Feb 9, 2012 at 10:57 AM, Todd Lipcon wrote:
>> Hi Konstantin,
>>
>> To be clear, th
so as to avoid changing too much code in a merge commit.
+ // Regardless of whether it's a new file or an updated file,
+ // update the block list.
updateBlocks(fsDir, addCloseOp, newFile);
-
break;
}
case OP_CLOSE: {
On Thu, Feb 9, 2012 at 1:20 PM, Todd Lip
Thanks Suresh. I committed the merge to the branch.
-Todd
On Thu, Feb 9, 2012 at 4:32 PM, Suresh Srinivas wrote:
> I looked at the merge. It looks good. +1.
>
> On Wed, Feb 8, 2012 at 9:08 PM, Todd Lipcon wrote:
>
>> The branch developed some new conflicts due to recen
egressions or
>> performance issues in the branch. If you plan to review the code changes or
>> the test plan, I ask that you please do so as soon as possible.
>>
>> Feedback is certainly welcome on this plan.
>>
>> Thanks a lot,
>> Aaron
>>
>> --
>> Aaron T. Myers
>> Software Engineer, Cloudera
>>
--
Todd Lipcon
Software Engineer, Cloudera
d, I think there are a couple easy wins that should bring
performance back in line with trunk, which I expect to complete by
early next week.
Please continue reviewing the branch so that when the optimizations
have been made, we can proceed with a merge.
-Todd
On Thu, Feb 23, 2012 at 10:27 AM, T
unless more blocker issues are discovered in the interim,
>> we merge this branch to trunk one week from today, i.e. Wednesday, February
>> 29th. During this time we will also execute the test plans described in the
>> test documents attached to HDFS-1623 to try to identify a
;mvn install" in the hadoop-hdfs-httpfs
directory on the branch on my machine.
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
On Wed, Feb 29, 2012 at 5:01 PM, Tsz Wo Sze wrote:
> Hi Todd,
>
> Then, why the trunk build did not suffer such problem?
Bad luck? Looks like a network blip - failure to download something
from archive.apache.org.
Todd
>
>
> ____
> From: Todd L
es, and as such would like to do it tomorrow.
>
> Here's a link to the latest build on Jenkins:
>
> https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/98/
>
> Please vote. We need a minimum of 3 +1s to proceed with the merge. Clearly
> I'm +1.
>
> Thanks,
> Aaron
>
> --
> Aaron T. Myers
> Software Engineer, Cloudera
--
Todd Lipcon
Software Engineer, Cloudera
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4184/
---
Review request for hadoop-hdfs and Todd Lipcon.
Summary
---
Posting patch
here vs the uglier
approach
hadoop-hdfs-project/hadoop-hdfs/src/main/native/hdfs.c
<https://reviews.apache.org/r/4184/#comment12240>
goofy empty clause. Other goofy indentation here.
- Todd
On 2012-03-05 21:00:55, Todd Lipcon wrote:
>
> -
automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4212/
> ---
>
> (Updated 2012-03-06 23:14:07)
>
>
> Review request for hadoop-hdfs and Todd Lipcon.
>
>
> Summary
> --
las
>> > merged protocol buffers/wire compatibility changes to 0.23. Merging
>> > Namenode HA into 0.23 will make it a significant release.
>> >
>> > I plan to complete this by next week after running some
>> nightly/regressions
>> > tests. Early next week, I plan to attach a 0.23 merge patch to HDFS-1623
>> > jira for people interested in reviewing the patch.
>> >
>> > Please vote.
>> >
>> > Regards,
>> > Suresh
>>
--
Todd Lipcon
Software Engineer, Cloudera
to be
HDFS-private, we drop steps 3-6 and use the protobuf RPC engine
directly. Doing this doesn't force our hand or limit our options in
the future -- should we want to add an alternate mechanism one can
always add the indirection layer down the road.
Thoughts?
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
is one of the essential prerequisites. Therefore, even
> if some protocols are not exposed to the users, and are just used within
> different internal components, we still cannot afford to compromise on wire
> compatibility on those interfaces.
>
> On Sat, Mar 17, 2012 at 3:31 PM, To
ndependent of each other and
>> wire compatibility is one of the essential prerequisites. Therefore, even
>> if some protocols are not exposed to the users, and are just used within
>> different internal components, we still cannot afford to compromise on wire
>> compatibility
To reply, visit:
> https://reviews.apache.org/r/4212/
> ---
>
> (Updated 2012-03-09 00:47:24)
>
>
> Review request for hadoop-hdfs and Todd Lipcon.
>
>
> Summary
> ---
>
> New patch for HDFS-28
on/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs/
> --
> Merged
> /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs:r1304063
>
> Propchange:
> hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc/
> --
> Merged
> /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc:r1304063
>
> Propchange:
> hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job/
> --
> Merged
> /hadoop/common/trunk/hadoop-mapreduce-project/src/webapps/job:r1304063
--
Todd Lipcon
Software Engineer, Cloudera
, do you think that my previous comment make sense?
>
Sure.
-Todd
>
> - Original Message -
> From: Todd Lipcon
> To: mapreduce-...@hadoop.apache.org; Tsz Wo Sze
> Cc: hdfs dev
> Sent: Thursday, March 22, 2012 3:49 PM
> Subject: Re: svn commit: r1304067 - in
>
as the
> commit message for both trunk and the branches.
+1. I never even use the svn merge command to merge between branches.
I just apply the patch separately to each branch. The svn metadata on
single-patch merges seems pretty much useless, best I can tell.
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
Auto failover (HDFS-3042)") in HDFS and Common JIRAs,
and an "auto-failover" component. I have not yet assigned these tags
to the open JIRAs but will do so tomorrow.
Thanks
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
ted at 2.x) yet, I've been
thinking of it more like 2.1. I also think, given we have PB in place
in both branches, it's likely we can maintain client compatibility
between 23 and 24 in the absence of anything major coming down the
pipe.
Proposal: is it possible to call the JIRA fixVersio
On Wed, Mar 28, 2012 at 11:59 AM, Aaron T. Myers wrote:
> On Wed, Mar 28, 2012 at 11:53 AM, Todd Lipcon wrote:
>
>> Proposal: is it possible to call the JIRA fixVersion "trunk", and then
>> when we branch off trunk to make a release, rename it at that point to
>
at we
currently call branch-23. When someone has something incompatible to
propose, then we can bother splitting it off..
Todd
--
Todd Lipcon
Software Engineer, Cloudera
r opinions.
Of course, I am +1.
Thanks
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
With the three requisite +1s, the merge vote passes. I'll merge this
to trunk in a moment.
Thanks
-Todd
On Sun, May 20, 2012 at 8:26 PM, Uma Maheswara Rao G
wrote:
> Great. +1 for merge.
> ____
> From: Todd Lipcon [t...@cloudera.com]
> Sen
nderstand correctly Hadoop 2.0 is not out of alpha, yet.
>>
>> Thoughts?
>>
>> -- Lars
>>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
--
Todd Lipcon
Software Engineer, Cloudera
tant
feature that we should have in the 2.0 line.
If I don't hear any objections in the next day or two, I'll go ahead and merge.
Thanks
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
Hearing no objections, I'll merge this this afternoon.
-Todd
On Wed, Jun 6, 2012 at 11:53 AM, Eli Collins wrote:
> +1 Thanks Todd
>
> On Wed, Jun 6, 2012 at 11:46 AM, Todd Lipcon wrote:
>> Hi folks,
>>
>> We merged automatic failover (HDFS-3042) into trunk
OK, I've merged to branch-2 and updated the various CHANGES.txt references.
Thanks
-Todd
On Thu, Jun 7, 2012 at 2:11 PM, Todd Lipcon wrote:
> Hearing no objections, I'll merge this this afternoon.
>
> -Todd
>
> On Wed, Jun 6, 2012 at 11:53 AM, Eli Collins wrote:
>
support
-format option. Contributed by Henry Robinson.
These test-only improvements make it easier to construct system tests
against realistic pseudo-distributed clusters.
All of the above patches have been baking in trunk for quite some time.
Thanks
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
Thanks. I'll merge these this this afternoon.
-Todd
On Thu, Jun 7, 2012 at 4:25 PM, Eli Collins wrote:
> +1
>
> Will be great to have the direct read optimization (thanks Henry) in 2.x.
>
> Thanks,
> Eli
>
> On Thu, Jun 7, 2012 at 2:51 PM, Todd Lipcon wrote:
>
Thanks all, I merged these as well as HDFS-3243 and HDFS-3514, some
small test fixes I forgot to add to my original list. Hopefully I got
all the CHANGES.txt, etc, right :)
-Todd
On Fri, Jun 8, 2012 at 11:53 AM, Todd Lipcon wrote:
> Thanks. I'll merge these this this afternoon.
>
>
onger
> this goes un-noticed, the more data could be lost if the NN cannot be
> recovered?
>
> Regards,
>
> James.
--
Todd Lipcon
Software Engineer, Cloudera
nning
>
> It looks like the PreCommit-Admin build job is not running.
> Can anyone give it a gentle nudge?
>
> Kihwal
>
>
--
Todd Lipcon
Software Engineer, Cloudera
;
> --
> Best regards,
>
>- Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein (via Tom White)
>
--
Todd Lipcon
Software Engineer, Cloudera
is very nearly ready to be merged
into trunk. I will continue to perform stress tests, but at this point
I am not aware of any deficiencies which would be considered a
blocker. If anyone has any questions about the code, design, or tests,
please feel free to chime in on HDFS-3077 or the relevant su
erging it under the HDFS umbrella.
>
> Thanks,
> --Konstantin
>
> On Wed, Sep 19, 2012 at 12:53 PM, Todd Lipcon wrote:
>> Hi all,
>>
>> Work has been progressing steadily for the last several months on the
>> HDFS-3077 (QuorumJournalManager) branch. The bra
o date as of yesterday.
Thanks
-Todd
> On Wed, Sep 19, 2012 at 12:53 PM, Todd Lipcon wrote:
>
>> Hi all,
>>
>> Work has been progressing steadily for the last several months on the
>> HDFS-3077 (QuorumJournalManager) branch. The branch is now "feature
>> c
thanks
> Sujee
>
>
> On Mon, Sep 10, 2012 at 4:18 PM, Sujee Maniyam wrote:
>
>> Thanks very much for all your input! Much appreciated...
>>
--
Todd Lipcon
Software Engineer, Cloudera
should be enough time for committers to
cast their votes. Per our by-laws, we need a minimum of three binding
+1 votes from committers.
I will start the voting with my own +1.
Thanks
-Todd
--
Todd Lipcon
Software Engineer, Cloudera
On Tue, Sep 25, 2012 at 4:02 PM, Todd Lipcon wrote:
> detail in a document attached to HDFS-3077 itself. The code itself has
> been contributed by me, Aaron, and Eli, but I'd be remiss not to also
I should actually amend the above: I had forgotten that some of the
early patches on
us, think it's currently
the best option for HA log storage.
If at some point in the future, the internal APIs have fully
stabilized (security, IPC, edit log streams, JournalManager, metrics,
etc) then we can pull it out at that time.
-Todd
> On Tue, Sep 25, 2012 at 4:58 PM, Eli Collins wro
o be undoing yourself with the above setup Konstantin. At
> our deploy, we can't do NetApp so calling them will never happen. If
> a problem in QJM, it's Apache HDFS, so it'll be fixed by the community
> -- hopefully w/ input by creators -- as any other issue would be fixed
> (or not).
>
> St.Ack
--
Todd Lipcon
Software Engineer, Cloudera
3077 code along with the code pulled from 3092 is fairly substrantial.
> The design is also fairly complex and involved.
> I would request that we postpone the merge for another week to give folks
> time to review this fully.
>
>
> sanjay
>
>
>
>
>
> On
t; for the @Test(expected = OutOfMemoryError.class) :)). Tested on Hadoop
> 1.0.2.
>
> Thanks in advance for your answers, pointers and advises.
>
> Benoit.
--
Todd Lipcon
Software Engineer, Cloudera
s plan to do the merge next monday.
>
> thanks
>
> sanjay
>
>
>
>
> On Sep 28, 2012, at 3:02 PM, Todd Lipcon wrote:
>
> > Hey Sanjay,
> >
> > While I understand it's substantial and complex code, the code and the
> > design doc have been
merge? Just like any other part
of HDFS, we can continue to _improve_ it after it is in trunk. There are
many features that we commit and then improve later on when someone comes
up with a way to simplify it.
To be clear about the current status of the vote: you are officially
vetoing the mer
1 - 100 of 1034 matches
Mail list logo
