+1 (binding)
sanjay
> On Mar 20, 2018, at 11:20 AM, Owen O'Malley wrote:
>
> All,
>
> Following our discussions on the previous thread (Merging branch HDFS-7240
> to trunk), I'd like to propose the following:
>
> * HDSL become a subproject of Hadoop.
> * HDSL will release separately from Hadoo
Owen,
Thanks for your proposal.
While I would have prefered to have HDSL in HDFS and also to be part
of Hadoop releases for the reasons stated earlier in this thread,
I am willing to accept your proposal as a compromise to move this forward.
Jitendra, Anu, Daryn, Andrew, Konstantine your thoughts
> On Mar 5, 2018, at 4:08 PM, Andrew Wang wrote:
>
> - NN on top HDSL where the NN uses the new block layer (Both Daryn and Owen
> acknowledge the benefit of the new block layer). We have two choices here
> ** a) Evolve NN so that it can interact with both old and new block layer,
> ** b) F
Joep, You raise a number of points:
(1) Ozone vs and object stores. “Some users would choose Ozone as that layer,
some might use S3, others GCS, or Azure, or something else”.
(2) How HDSL/Ozone fits into Hadoop and whether it is necessary.
(3) You raise the release issue which we will respond in
ge Layer (HDSL), which
>> is
>>> a distributed, replicated block layer.
>>>The old HDFS namespace and NN can be connected to this new block
>> layer
>>> as we have described in HDFS-10419.
>>>We also introduce a key-value namespace called Ozone built on HDSL.
&g
>>>We also introduce a key-value namespace called Ozone built on HDSL.
>>>
>>>The code is in a separate module and is turned off by default. In a
>>> secure setup, HDSL and Ozone daemons cannot be started.
>>>
>>
Konstantine
Thanks for your feedback and comments over the last few months. Have we
addressed all your issues and concerns?
sanjay
> On Feb 13, 2018, at 6:28 PM, sanjay Radia wrote:
>
> Sorry the formatting got messed by my email client. Here it is again
>
>
Sorry the formatting got messed by my email client. Here it is again
Dear
Hadoop Community Members,
We had multiple community discussions, a few meetings in smaller groups and
also jira discussions with respect to this thread. We express our gratitude for
participation and valuable commen
Dear Hadoop Community Members,
We had multiple community discussions, a few meetings in smaller groups and
also jira discussions with respect to this thread. We express our gratitude for
participation and valuable comments. The key questions raised were following
How the new block storage l
Sanjay Radia created HDFS-12952:
---
Summary: Change OzoneFS's semtics to allow readers to see file
content while being written
Key: HDFS-12952
URL: https://issues.apache.org/jira/browse/HDFS-
Konstantine,
Thanks for your comments, questions and feedback. I have attached a document
to the HDFS-7240 jira
that explains a design for scaling HDFS and how Ozone paves the way towards
the full solution.
https://issues.apache.org/jira/secure/attachment/12895963/HDFS%20Scalability%20and%2
Erik
Great stuff.
BTW did you build on top of the “simulated data nodes” in HDFS which has a way
to storing only the length of data (but not real data)? That work allowed
supplementing with a matching editsLog for the NN. Your approach of using a
real image has the advantage of being able to
[
https://issues.apache.org/jira/browse/HDFS-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjay Radia resolved HDFS-8075.
Resolution: Duplicate
Closed: duplicate of HDFS-5223
> Revist layout vers
Sanjay Radia created HDFS-8075:
--
Summary: Revist layout version
Key: HDFS-8075
URL: https://issues.apache.org/jira/browse/HDFS-8075
Project: Hadoop HDFS
Issue Type: Bug
Components
> On Mar 5, 2015, at 3:21 PM, Siddharth Seth wrote:
>
> 2) Simplification of configs - potentially separating client side configs
> and those used by daemons. This is another source of perpetual confusion
> for users.
+ 1 on this.
sanjay
Andrew
Thanks for bringing up the issue of moving to Java8. Java8 is important
However, I am not seeing a strong motivation for changing the major number.
We can go to Java8 in the 2.series.
The classpath issue for Hadoop-11656 is too minor to force a major number
change (no pun intended).
L
Sanjay Radia created HDFS-7745:
--
Summary: HDFS should have its own daemon command and not rely on
the one in common
Key: HDFS-7745
URL: https://issues.apache.org/jira/browse/HDFS-7745
Project: Hadoop
transparent encryption in a 2.x release
for pubic consumption, let us at least complete item 1 (ie distcp and cp) and
the flag to turn this feature on/of.
This is a great work; thanks team for contributing this important feature.
sanjay
On Aug 14, 2014, at 1:05 AM, sanjay Radia wrote:
> Whil
+1
sanjay
>
> On Fri, Aug 8, 2014 at 7:57 PM, Karthik Kambatla wrote:
>> I have put together this proposal based on recent discussion on this topic.
>>
>> Please vote on the proposal. The vote runs for 7 days.
>>
>> 1. Migrate from subversion to git for version control.
>> 2. Force-push to
While I was originally skeptical of transparent encryption, I like the value
proposition of transparent encryption. HDFS has several layers, protocols and
tools. While the HDFS core part seems to be well done in the Jira, inserting
the matching transparency in the other tools or protocols need
I am at the meeting via the phone but can barely follow the audio. Also looks
like the google plus hangout posted in the invite not been enabled - I am the
only person on the hangout.
sanjay
On Jul 2, 2014, at 6:37 PM, Konstantin Boudnik wrote:
> We'd like to invite you to the
>Consens
On Jun 21, 2014, at 8:01 AM, Andrew Wang wrote:
> This is why I'd like to keep my original proposal on the table: keep going
> with branch-2 in the near term, while working towards a JDK8-based Hadoop 3
> by April next year. It doesn't need to be a big bang release either. I'd be
> delighted if
+1 binding
Verified binaries, ran from binary on single node cluster. Tested some HDFS
clis and wordcount.
sanjay
On Apr 7, 2014, at 9:52 AM, Suresh Srinivas wrote:
> +1 (binding)
>
> Verified the signatures and hashes for both src and binary tars. Built from
> the source, the binary distrib
On Thu, Apr 3, 2014 at 4:55 PM, Tsuyoshi OZAWA wrote:
> Hi,
>
> Ran tests and confirmed that some tests(TestSymlinkLocalFSFileSystem)
> fail.
The log of the test failure is as follows:
>
> https://gist.github.com/oza/9965197
>
> Should we fix or disable the feature?
>
Symlinks is still not compl
On Oct 3, 2013, at 12:17 PM, Milind Bhandarkar wrote:
> Exec Summary: For the last couple of months, we, at Pivotal, along with a
> couple of folks in the community have been working on making Namespace
> implementation in the namenode pluggable. We have demonstrated that it can
> be done without
On Oct 6, 2013, at 5:58 PM, Milind Bhandarkar wrote:
> Vinod,
>
> I have received a few emails about concerns that this effort somehow
> conflicts with federated namenodes. Most of these emails are from folks
> who are directly or remotely associated with Hortonworks.
>
> Three weeks ago, I sen
There are a number of issues (some minor, some more than minor).
GA is close and we are are still in discussion on the some of them; while I
believe we will close on these very very shortly, code change like this so
close to GA is dangerous.
I suggest we do the following:
1) Disable Symlinks in
+1 for naming the new branch 2.2.0
sanjay
On Oct 1, 2013, at 4:55 PM, Suresh Srinivas wrote:
> (This time copying all the lists)
>
> I am +1 for naming the new branch 2.2.0.
>
>
> On Tue, Oct 1, 2013 at 4:15 PM, Arun C Murthy wrote:
>
>> Guys,
>>
>> I took a look at the content in 2.1.2-bet
Sanjay Radia created HDFS-4691:
--
Summary: Disallow .reserved path component, given that HDFS-4434
does not allow /.reserved
Key: HDFS-4691
URL: https://issues.apache.org/jira/browse/HDFS-4691
Project
Sanjay Radia created HDFS-4606:
--
Summary: HDFS API to move file replicas to caller's location
Key: HDFS-4606
URL: https://issues.apache.org/jira/browse/HDFS-4606
Project: Hadoop HDFS
Issue
On Mar 1, 2013, at 1:57 PM, Konstantin Shvachko wrote:
> Commitment is a good thing.
> I think the two builds that I proposed are a prerequisite for Win support.
> If we commit windows patch people will start breaking it the next day.
> Which we wont know without the nightly build and wont be abl
+1
Java has done the bulk of the work in making Hadoop multi-platform.
Windows specific code is a tiny percentage of the code.
Jeninks support for windows is going help us keep the platform portable going
forward.
I expect that the vast majority of new commits have no problems. I propose
that we
On Feb 26, 2013, at 1:51 PM, Eli Collins wrote:
> it doesn't seem right to hold up 347 up for Windows support given that
> Windows support has not been merged to trunk yet, is not in any Apache
> release, etc. Personally I don't like establishing the precedent here
> that we can hold up a merge d
On Feb 20, 2013, at 5:12 PM, Aaron T. Myers wrote:
> On Wed, Feb 20, 2013 at 4:29 PM, Chris Douglas wrote:
>
>> Given that HDFS-347 is a strictly better approach, once committed,
>> there will be ample motivation to add support for other OSes and
>> remove HDFS-2246 entirely. Nobody is confused
[
https://issues.apache.org/jira/browse/HDFS-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjay Radia resolved HDFS-4256.
Resolution: Fixed
Target Version/s: 1.2.0
> Backport concatenation of files int
Sanjay Radia created HDFS-4251:
--
Summary: NN connections can use up all fds leaving none for
rolling journal files
Key: HDFS-4251
URL: https://issues.apache.org/jira/browse/HDFS-4251
Project: Hadoop
Sanjay Radia created HDFS-4040:
--
Summary: Fix/cleanup viewfs test setup
Key: HDFS-4040
URL: https://issues.apache.org/jira/browse/HDFS-4040
Project: Hadoop HDFS
Issue Type: Bug
On Oct 9, 2012, at 1:44 PM, sanjay Radia wrote:
> I have posted a comment on 3077 listing two areas that i would like to fix
> in the QJM.
> If we agree on that we can merge now and fix those two later.
> Todd has volunteered to fix one and I or suresh can fix the other.
>
&g
Sanjay Radia created HDFS-4028:
--
Summary: QJM: Merge newEpoch and prepareRecovery
Key: HDFS-4028
URL: https://issues.apache.org/jira/browse/HDFS-4028
Project: Hadoop HDFS
Issue Type: Sub-task
I have posted a comment on 3077 listing two areas that i would like to fix in
the QJM.
If we agree on that we can merge now and fix those two later.
Todd has volunteered to fix one and I or suresh can fix the other.
sanjay
one number and I'm not going anywhere! ;-)
>
> Thanks
> -Todd
>
> On Fri, Sep 28, 2012 at 12:06 PM, sanjay Radia wrote:
>> Suresh and I are still reviewing this design and patch.
>> The 3077 code along with the code pulled from 3092 is fairly substrantial.
Suresh and I are still reviewing this design and patch.
The 3077 code along with the code pulled from 3092 is fairly substrantial. The
design is also fairly complex and involved.
I would request that we postpone the merge for another week to give folks time
to review this fully.
sanjay
I am in favor of 3077 being part of HDFS.
- It is one of several way of configuring HDFS HA (shared storage is an
option or if we extend the BackupNode a little, then it will be another option
for HA.)
- I believe I can use part of 3092/3077 to move journals to the Datanodes
down th
Sanjay Radia created HDFS-3630:
--
Summary: Modify TestPersistBlocks to use both flush and hflush
Key: HDFS-3630
URL: https://issues.apache.org/jira/browse/HDFS-3630
Project: Hadoop HDFS
Issue
[
https://issues.apache.org/jira/browse/HDFS-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjay Radia resolved HDFS-3424.
Resolution: Fixed
Target Version/s: HADOOP-1-Windows
Committed to windows branch. Thanks
We will have a Hdfs Contributors meetup at the Hadoop summit.
Details:
http://www.meetup.com/Hadoop-Contributors/events/60841502/
sanjay
Umbrella Jira for HDFS-HA Phase 2
-
Key: HDFS-3278
URL: https://issues.apache.org/jira/browse/HDFS-3278
Project: Hadoop HDFS
Issue Type: New Feature
Reporter: Sanjay Radia
Assignee
Type: New Feature
Reporter: Sanjay Radia
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see
Type: Improvement
Reporter: Sanjay Radia
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see
On Tue, Mar 20, 2012 at 5:37 PM, Eli Collins wrote:
>
>
> Append introduces non-trivial design and code complexity, which is not
> worth the cost if we don't have real users.
The bulk of the complexity of HDFS-265 ("the new Append") was around
Hflush, concurrent readers, the pipeline etc. The co
approx 15 folks - please send me (san...@hortonworks.com)
an email if you are interested in attending so that I can reserve a spot and
confirm. Minutes will be posted on HDFS-1623.
sanjay
On Feb 21, 2012, at 9:16 PM, sanjay Radia wrote:
> Hello,
> We would like organize a meeting to d
Hello,
We would like organize a meeting to discuss the 2nd phase of HDFS HA
development.
Right now the HA branch supports HOT-Failover, except that it is manual
failover. We are now moving into a phase to implement automatic failover.
Please see HDFS-1623 for more details.
Significant enhanceme
Make FSDataset in Datanode Pluggable
Key: HDFS-2907
URL: https://issues.apache.org/jira/browse/HDFS-2907
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
Enable protobuf rpc for ClientNamenodeProtocol
--
Key: HDFS-2669
URL: https://issues.apache.org/jira/browse/HDFS-2669
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Sanjay Radia
Reporter: Sanjay Radia
Assignee: Sanjay Radia
Attachments: protoClientNNTranslators1.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators
ClientNameNodeProtocol in Protocol Buffers
---
Key: HDFS-2597
URL: https://issues.apache.org/jira/browse/HDFS-2597
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Sanjay Radia
ClientDatanodeProtocol in Protocol Buffers
--
Key: HDFS-2598
URL: https://issues.apache.org/jira/browse/HDFS-2598
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Sanjay Radia
HDFS Client Data Types in Protocol Buffers
--
Key: HDFS-2479
URL: https://issues.apache.org/jira/browse/HDFS-2479
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Sanjay Radia
HDFS Protocols in Protocol Buffers
--
Key: HDFS-2478
URL: https://issues.apache.org/jira/browse/HDFS-2478
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
Assignee
/HDFS-2400
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure
://issues.apache.org/jira/browse/HDFS-2399
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
Assignee: Sanjay Radia
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA
Issue Type: Improvement
Reporter: Sanjay Radia
Assignee: Sanjay Radia
This take advantage of HADOOP-7524 and allow separation of datatypes
(HDFS-2181) jira to be completed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http
RW Http access to HDFS
--
Key: HDFS-2284
URL: https://issues.apache.org/jira/browse/HDFS-2284
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
Assignee: Tsz Wo (Nicholas), SZE
Reporter: Sanjay Radia
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Seperate HDFS wire protocol data types
--
Key: HDFS-2181
URL: https://issues.apache.org/jira/browse/HDFS-2181
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
I have updated meetup:
http://www.meetup.com/Hadoop-Contributors/events/23890191/
sanjay
On Jun 28, 2011, at 4:27 PM, Sanjay Radia wrote:
On Jun 28, 2011, at 11:17 AM, Sanjay Radia wrote:
We have a room confirmed from 10-3pm on Friday July 1st at Yahoo's
sunnyvale campus.
Will updat
-task
Reporter: Sanjay Radia
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
HDFS tests for viewfs
-
Key: HDFS-1911
URL: https://issues.apache.org/jira/browse/HDFS-1911
Project: Hadoop HDFS
Issue Type: Test
Reporter: Sanjay Radia
Assignee: Sanjay Radia
Fix For
upgrads to the federation version, right?
Cos
On Tue, Apr 26, 2011 at 17:26, Sanjay Radia
wrote:
Changes to the code base
- The fundamental code change is to extend the notion of block id
to now
include a block pool id.
- The NN had little change, the protocols did change to include
the
On Apr 25, 2011, at 2:36 PM, Doug Cutting wrote:
A couple of questions:
1. Can you please describe the significant advantages this approach
has
over a symlink-based approach?
It seems to me that one could run multiple namenodes on separate boxes
and run multile datanode processes per storage
On Mar 14, 2011, at 10:57 AM, Sanjay Radia wrote:
On Mar 12, 2011, at 8:43 AM, Allen Wittenauer wrote:
To me, this series of changes looks like it is going to make
running a grid much much harder for very little benefit. In
particular, I don't see the difference between ru
On Mar 12, 2011, at 8:43 AM, Allen Wittenauer wrote:
On Mar 3, 2011, at 2:41 PM, Suresh Srinivas wrote:
We have started pushing changes for namenode federation in to the
feature branch HDFS-1052. The work items are created as subtask of
the jira HDFS-1052 and are based on the design docum
High Availability Framework for HDFS NN
---
Key: HDFS-1623
URL: https://issues.apache.org/jira/browse/HDFS-1623
Project: Hadoop HDFS
Issue Type: New Feature
Reporter: Sanjay Radia
: Sanjay Radia
Umbrella Jira for improved HBase support in HDFS
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[
https://issues.apache.org/jira/browse/HDFS-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjay Radia resolved HDFS-1491.
Resolution: Fixed
Committed on Nov 10.
> Update Hdfs to match the change of methods f
Nigel
At my previous company we simply checked-in the .classpath file and
the .project file rather then generate it
through an ant command.
Folks who were not using eclipse ignored it.
Should we consider such an approach for eclipse in the future. It is
much easier to manage.
sanjay
On
/browse/HDFS-1491
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
Assignee: Sanjay Radia
Fix For: 0.22.0
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the
be found at:
- Meetup home page: http://www.meetup.com/Hadoop-Contributors/
- May 6th registration:
http://www.meetup.com/Hadoop-Contributors/calendar/13249370/
thanks
Sanjay Radia
Hadoop Team, Yahoo!
orter: Sanjay Radia
The naming and handling of NN's fsImage and edit logs can be significantly
improved resulting simpler and more robust code.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Write HDFS wire protocols in AVRO IDL
-
Key: HDFS-1069
URL: https://issues.apache.org/jira/browse/HDFS-1069
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Sanjay Radia
NN Availability - umbrella Jira
---
Key: HDFS-1064
URL: https://issues.apache.org/jira/browse/HDFS-1064
Project: Hadoop HDFS
Issue Type: New Feature
Reporter: Sanjay Radia
This is an umbrella jira
Issue Type: New Feature
Affects Versions: 0.22.0
Reporter: Sanjay Radia
Assignee: Sanjay Radia
Fix For: 0.22.0
This jira proposes a client side mount table to allow application-centric (or
job-centric) filesystem views.
--
This message is automatically
Reporter: Sanjay Radia
Assignee: Sanjay Radia
Fix For: 0.22.0
The HDFS Name service currently uses a single Namenode which limits its
scalability. This is a master jira to track sub-jiras to address this problem.
--
This message is automatically generated by
Issue Type: Improvement
Reporter: Sanjay Radia
Fix For: 0.22.0
The DataNodeCluster (built using the MiniDFSCluster) allows injection of blocks
only if the data nodes are simulated. Extend the implementation to allow block
injection for DataNodeClusters that have real
: Sanjay Radia
Attachments: buildError-hdfs.patch
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Reporter: Sanjay Radia
Assignee: Sanjay Radia
Fix For: 0.22.0
HADOOP-6223 adds a new file system interface (potentially called
AbstractFileSystem).
Add an implementation for it for the HDFS file system.
--
This message is automatically generated by JIRA.
-
You
Issue Type: Bug
Reporter: Sanjay Radia
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Reporter: Sanjay Radia
Assignee: Sanjay Radia
Add support for FileContext (linked jira)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
88 matches
Mail list logo
