Re: ReadOnly WebHDFS

2015-11-06 Thread Laxman Ch
Thanks for your response Chris. >> I'm curious how this is a new problem after migration from HttpFs to WebHDFS With HttpFs, gateway admin users are allowed to proxy some limited set of users. They are not allowed to mock a root/dfsadmin user and do some fatal stuff ("rmr /"). With WebHDFS, thats

Re: ReadOnly WebHDFS

2015-11-06 Thread Chris Nauroth
Hello Laxman, I'm curious how this is a new problem after migration from HttpFs to WebHDFS. With the HttpFs deployment architecture, were you somehow proxying only the read-only operations? I would think this kind of thing could be achieved by writing a custom authentication filter, deploying th

ReadOnly WebHDFS

2015-11-06 Thread Laxman Ch
Hi, We run a cluster with security set to simple. Also, to some users, we had provided http access to HDFS via HttpFS gateways. However, this is not scaling and we are suffering from HttpFs gateway choking problem. So, we wanted to enable WebHDFS directly on hadoop. But this brings in the problem