webhdfs does not honour its own cookies ---------------------------------------
Key: HDFS-3198 URL: https://issues.apache.org/jira/browse/HDFS-3198 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.23.1 Reporter: bc Wong WebHdfs gives out cookies. But when the client passes them back, it'd sometimes reject them and return a HTTP 401 instead. ("Sometimes" as in after a restart.) The interesting thing is that if the client doesn't pass the cookie back, WebHdfs will be totally happy. The correct behaviour should be to ignore the cookie if it looks invalid, and attempt to proceed with the request handling. I haven't tried HttpFs to see whether it handles restart better. Reproducing it with curl: {noformat} #################################################### ## Initial curl. Storing cookie to file. #################################################### [root@vbox2 ~]# curl -c /tmp/webhdfs.cookie -i 'http://localhost:50070/webhdfs/v1/?op=LISTSTATUS&user.name=bcwalrus' HTTP/1.1 200 OK Content-Type: application/json Expires: Thu, 01-Jan-1970 00:00:00 GMT Set-Cookie: hadoop.auth="u=bcwalrus&p=bcwalrus&t=simple&e=1333614686366&s=z2w5xpFlufnnEoOHxVRiXqxwtqM=";Path=/ Content-Length: 597 Server: Jetty(6.1.26) {"FileStatuses":{"FileStatus":[ {"accessTime":0,"blockSize":0,"group":"supergroup","length":0,"modificationTime":1333577906198,"owner":"mapred","pathSuffix":"tmp","permission":"1777","replication":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"group":"supergroup","length":0,"modificationTime":1333577511848,"owner":"hdfs","pathSuffix":"user","permission":"1777","replication":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"group":"supergroup","length":0,"modificationTime":1333428745116,"owner":"mapred","pathSuffix":"var","permission":"755","replication":0,"type":"DIRECTORY"} ]}} #################################################### ## Another curl. Using the cookie jar. #################################################### [root@vbox2 ~]# curl -b /tmp/webhdfs.cookie -i 'http://localhost:50070/webhdfs/v1/?op=LISTSTATUS&user.name=bcwalrus' HTTP/1.1 200 OK Content-Type: application/json Content-Length: 597 Server: Jetty(6.1.26) {"FileStatuses":{"FileStatus":[ {"accessTime":0,"blockSize":0,"group":"supergroup","length":0,"modificationTime":1333577906198,"owner":"mapred","pathSuffix":"tmp","permission":"1777","replication":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"group":"supergroup","length":0,"modificationTime":1333577511848,"owner":"hdfs","pathSuffix":"user","permission":"1777","replication":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"group":"supergroup","length":0,"modificationTime":1333428745116,"owner":"mapred","pathSuffix":"var","permission":"755","replication":0,"type":"DIRECTORY"} ]}} #################################################### ## Restart NN. #################################################### [root@vbox2 ~]# /etc/init.d/hadoop-hdfs-namenode restartStopping Hadoop namenode: [ OK ] stopping namenode Starting Hadoop namenode: [ OK ] starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-vbox2.out #################################################### ## Curl using cookie jar gives error. #################################################### [root@vbox2 ~]# curl -b /tmp/webhdfs.cookie -i 'http://localhost:50070/webhdfs/v1/?op=LISTSTATUS&user.name=bcwalrus' HTTP/1.1 401 org.apache.hadoop.security.authentication.util.SignerException: Invalid signature Content-Type: text/html; charset=iso-8859-1 Set-Cookie: hadoop.auth=;Path=/;Expires=Thu, 01-Jan-1970 00:00:00 GMT Cache-Control: must-revalidate,no-cache,no-store Content-Length: 1520 Server: Jetty(6.1.26) <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/> <title>Error 401 org.apache.hadoop.security.authentication.util.SignerException: Invalid signature</title> </head> <body><h2>HTTP ERROR 401</h2> <p>Problem accessing /webhdfs/v1/. Reason: <pre> org.apache.hadoop.security.authentication.util.SignerException: Invalid signature</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/> ... #################################################### ## Curl without cookie jar is ok. #################################################### [root@vbox2 ~]# curl -i 'http://localhost:50070/webhdfs/v1/?op=LISTSTATUS&user.name=bcwalrus' HTTP/1.1 200 OK Content-Type: application/json Expires: Thu, 01-Jan-1970 00:00:00 GMT Set-Cookie: hadoop.auth="u=bcwalrus&p=bcwalrus&t=simple&e=1333614995947&s=IXSvPIDbNrqmZryivGeoey6Kjwo=";Path=/ Content-Length: 597 Server: Jetty(6.1.26) {"FileStatuses":{"FileStatus":[ {"accessTime":0,"blockSize":0,"group":"supergroup","length":0,"modificationTime":1333577906198,"owner":"mapred","pathSuffix":"tmp","permission":"1777","replication":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"group":"supergroup","length":0,"modificationTime":1333577511848,"owner":"hdfs","pathSuffix":"user","permission":"1777","replication":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"group":"supergroup","length":0,"modificationTime":1333428745116,"owner":"mapred","pathSuffix":"var","permission":"755","replication":0,"type":"DIRECTORY"} ]}} {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira