[
https://issues.apache.org/jira/browse/PIG-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13858265#comment-13858265
]
Cheolsoo Park commented on PIG-3617:
------------------------------------
[~nezihyigitbasi], sorry for the late reply.
I see why you propose implementing a DataStorage interface for HBase, but that
sounds like an overkill for this problem. Do you think that will be useful in
the future for other things?
In addition, how about handling an hbase prefix in MapReduceLauncher rather
than in FileLocalizer? PIG-3592 fixes a similar issue, and it does in
MapReducerLauncher. I think it is better to do storage-specific things in
Launcher than in FileLocalizer because the latter currently assumes file system.
Feel free to disagree with me. Thanks!
> problem with temp file deletion in MAPREDUCE operator
> -----------------------------------------------------
>
> Key: PIG-3617
> URL: https://issues.apache.org/jira/browse/PIG-3617
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.12.0
> Reporter: Nezih Yigitbasi
>
> Hi all,
> When I run a native MR job with the MAPREDUCE keyword and store the
> intermediate data in HBase with:
> stored = MAPREDUCE 'my.jar'
> STORE x INTO 'hbase://temp_table'
> USING
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('hbase_schema')
> .... and the rest ....;
> Pig tries to delete the temp files, which in this case has an HBase path, and
> fails with the exception:
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: file:hbase:/temp_table
> at org.apache.hadoop.fs.Path.initialize(Path.java:148)
> at org.apache.hadoop.fs.Path.<init>(Path.java:126)
> at
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.isContainer(HDataStorage.java:197)
> at
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.asElement(HDataStorage.java:128)
> at org.apache.pig.impl.io.FileLocalizer.delete(FileLocalizer.java:415)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:419)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
