Re: Proxy for HDFS

read the documentation. From: Rahul Shrivastava To: Rajiv Chittajallu Cc: "hdfs-dev@hadoop.apache.org" Sent: Monday, February 9, 2015 9:53 PM Subject: Re: Proxy for HDFS Hi Rajiv, At runtime, can the user's program provides its credentials( username, passwo

[jira] [Created] (HDFS-7764) DirectoryScanner should cancel the future tasks when #compileReport throws exception

Rakesh R created HDFS-7764: -- Summary: DirectoryScanner should cancel the future tasks when #compileReport throws exception Key: HDFS-7764 URL: https://issues.apache.org/jira/browse/HDFS-7764 Project: Hadoop

[jira] [Created] (HDFS-7763) fix zkfc hung issue due to not catching exception in a corner case

Liang Xie created HDFS-7763: --- Summary: fix zkfc hung issue due to not catching exception in a corner case Key: HDFS-7763 URL: https://issues.apache.org/jira/browse/HDFS-7763 Project: Hadoop HDFS I

Re: Proxy for HDFS

Hi Rajiv, At runtime, can the user's program provides its credentials( username, password etc) and hdfs can verify it with the ACL entries ( username , ofcourse password cannot be stored) to grant/deny it access to the file? So, the basic question is, can the user/group be provided at runtime rath

[jira] [Created] (HDFS-7762) Add more unit tests to TestDFSOutputStream

Li Bo created HDFS-7762: --- Summary: Add more unit tests to TestDFSOutputStream Key: HDFS-7762 URL: https://issues.apache.org/jira/browse/HDFS-7762 Project: Hadoop HDFS Issue Type: Test Compone

[jira] [Created] (HDFS-7761) cleanup unnecssary code logic in LocatedBlock

Yi Liu created HDFS-7761: Summary: cleanup unnecssary code logic in LocatedBlock Key: HDFS-7761 URL: https://issues.apache.org/jira/browse/HDFS-7761 Project: Hadoop HDFS Issue Type: Improvement A

[jira] [Created] (HDFS-7760) Further update user doc for truncate

Yi Liu created HDFS-7760: Summary: Further update user doc for truncate Key: HDFS-7760 URL: https://issues.apache.org/jira/browse/HDFS-7760 Project: Hadoop HDFS Issue Type: Sub-task Compone

[jira] [Created] (HDFS-7759) Provide existence-of-a-second-file implementation for pinning blocks on Datanode

zhaoyunjiong created HDFS-7759: -- Summary: Provide existence-of-a-second-file implementation for pinning blocks on Datanode Key: HDFS-7759 URL: https://issues.apache.org/jira/browse/HDFS-7759 Project: Had

[jira] [Created] (HDFS-7758) Retire FsDatasetSpi#getVolumes() and use FsDatasetSpi#getVolumeRefs() instead

Lei (Eddy) Xu created HDFS-7758: --- Summary: Retire FsDatasetSpi#getVolumes() and use FsDatasetSpi#getVolumeRefs() instead Key: HDFS-7758 URL: https://issues.apache.org/jira/browse/HDFS-7758 Project: Hado

Re: Proxy for HDFS

SOCKS proxies TCP connections. It wouldn't understand L7 traffic. New HDFS ACL feature[1] would provide additional controls. If there is a need beyond that, it would be better as a enhancement request in HDFS than building a proxy. [1] http://hadoop.apache.org/docs/current/hadoop-project-dist/

Re: Proxy for HDFS

Hi Rajiv, Thanks again for quick reply. The use case is as follows. Please let me know how can i use HDFS RPC+SOCKS proxy to achieve below. 1. Client connect to HDFS proxy using HDFS driver ( not HTTPFs or WebHDFS).Proxy that sits between client and HDFS server. 2. Client request to access a fil

Build failed in Jenkins: Hadoop-Hdfs-trunk #2031

See Changes: [jing9] HDFS-7743. Code cleanup of BlockInfo and rename BlockInfo to BlockInfoContiguous. Contributed by Jing Zhao. [kasha] YARN-2990. FairScheduler's delay-scheduling always waits for node-local and rack-local delays,

Hadoop-Hdfs-trunk - Build # 2031 - Failure

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2031/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 6460 lines...] [INFO] [INFO] --- maven-antrun-plugin:1.7

Re: Proxy for HDFS

Rahul, If a client can use HDFS RPC why is a Proxy required? Are the clients not allowed to reach data nodes directly? WebHDFS + Apache Traffic server or HDFS RPC + SOCK Proxy should work. If you can share a use case, it would probably help. -rajive - Original Message - From: Rahul

Re: Proxy for HDFS

Thanks Rajiv. I did look into HttpFs before but i wanted a build a proxy at HDFS layer. This is specific for clients which do not use HTTP ( i.e. HttpFs or webHDFS) to talk to the HDFS cluster. That includes clients which uses HDFS driver to talk to HDFS cluster. thanks Rahul On Mon, Feb 9, 2

Re: Proxy for HDFS

https://github.com/apache/hadoop/tree/branch-2.6/hadoop-hdfs-project/hadoop-hdfs-httpfs - Original Message - From: Rahul Shrivastava To: hdfs-dev@hadoop.apache.org Cc: Sent: Monday, February 9, 2015 2:35 PM Subject: Proxy for HDFS Hi, I have looking in HDFS code base ( 2.6.0) for th

[jira] [Created] (HDFS-7757) Misleading error messages in FSImage.java

Arpit Agarwal created HDFS-7757: --- Summary: Misleading error messages in FSImage.java Key: HDFS-7757 URL: https://issues.apache.org/jira/browse/HDFS-7757 Project: Hadoop HDFS Issue Type: Bug

Proxy for HDFS

Hi, I have looking in HDFS code base ( 2.6.0) for the last couple of days for any possible proxy that we could utilize to create a proxy for HDFS. Is there any hook within HDFS to build a proxy around that. thanks Rahul

[jira] [Created] (HDFS-7756) DatanodeInfoWithStorage should be tagged Private

Ted Yu created HDFS-7756: Summary: DatanodeInfoWithStorage should be tagged Private Key: HDFS-7756 URL: https://issues.apache.org/jira/browse/HDFS-7756 Project: Hadoop HDFS Issue Type: Task

Re: Erratic Jenkins behavior

Sorry I don¹t have much insight into this either. Our Jenkins setup is a bit of a black box to me. Too few of us have access to the build hosts and it makes debugging difficult. Perhaps we should grant access to all committers. On 2/9/15, 11:29 AM, "Colin P. McCabe" wrote: >I'm sorry, I don't

[jira] [Resolved] (HDFS-4718) TestHDFSCLI regexps reject valid user names

[ https://issues.apache.org/jira/browse/HDFS-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dasha Boudnik resolved HDFS-4718. - Resolution: Duplicate > TestHDFSCLI regexps reject valid user names > -

Re: Erratic Jenkins behavior

I'm sorry, I don't have any insight into this. With regard to HADOOP-11084, I thought that $BUILD_URL would be unique for each concurrent build, which would prevent build artifacts from getting mixed up between jobs. Based on the value of PATCHPROCESS that Kihwal posted, perhaps this is not the c

[jira] [Resolved] (HDFS-5998) TestHDFSCLI tests will fail if machine user names are not the combination of alphanumeric

[ https://issues.apache.org/jira/browse/HDFS-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dasha Boudnik resolved HDFS-5998. - Resolution: Duplicate > TestHDFSCLI tests will fail if machine user names are not the combination o

[jira] [Resolved] (HDFS-7652) Process block reports for erasure coded blocks

[ https://issues.apache.org/jira/browse/HDFS-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7652. - Resolution: Fixed > Process block reports for erasure coded blocks > --