[
https://issues.apache.org/jira/browse/PIG-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13722979#comment-13722979
]
Cheolsoo Park commented on PIG-3400:
------------------------------------
Hi Rohini, thank you for the comment.
In fact, that's something that I was debating myself. I intentionally didn't do
that for PWD and CD. I wasn't sure whether it makes sense to cd to a s3 path.
Given constructing a new instance of DataStorage is not free, I wanted to avoid
it when not useful. Does this make sense? I can certainly document it though.
> FS commands do not work with S3 paths
> -------------------------------------
>
> Key: PIG-3400
> URL: https://issues.apache.org/jira/browse/PIG-3400
> Project: Pig
> Issue Type: Bug
> Components: grunt
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.12
>
> Attachments: PIG-3400-2.patch, PIG-3400.patch
>
>
> To reproduce issues, run the following commands w/ S3 paths:
> {code}
> pig -e 'ls s3://<path>'
> pig -e 'mkdir s3://<path>'
> pig -e 'cp s3://<path1> s3://<path2>'
> pig -e 'mv s3://<path1> s3://<path2>'
> pig -e 'copyToLocal s3://<path>/<file> .'
> pig -e 'copyFromLocal <file> s3://<path>'
> {code}
> As of now, none of these commands works in Apache Pig, whereas they do in EMR
> Pig. The problem is that in GruntParser, DataStorage is constructed using the
> default file system provided by configuration, and s3 paths are not
> recognized.
> Instead, we should construct DataStorage based on the given URL. For example,
> {code}
> DataStorage dfs = new HDataStorage(new Path(path).toUri(), mConf);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
