[
https://issues.apache.org/jira/browse/HIVE-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13024891#comment-13024891
]
Namit Jain commented on HIVE-2126:
----------------------------------
It might be simpler if the HiveInputFormat (or a new interface which extends
InputFommat) adds
this new method.
All Hive input formats will implement the above interface. The default
implementation does nothing,
You dont need code like below:
if
(partDesc.getInputFileFormatClass().equals(SymlinkTextInputFormat.class)) {
//change to TextInputFormat
You always call a new method, which is a no-op for all other input formats
right now.
> Hive's symlink text input format should be able to work with
> ComineHiveInputFormat
> ----------------------------------------------------------------------------------
>
> Key: HIVE-2126
> URL: https://issues.apache.org/jira/browse/HIVE-2126
> Project: Hive
> Issue Type: Improvement
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: HIVE-2126.1.patch
>
>
> at compile time, if a partition's file format is SymlinkTextInputFormat, will
> replace the symlink path with paths in the symlink file. This way, it will
> work with Hive's HiveCombineFileInputFormat.
> The reason we are doing it at compile time is because:
> 1) At run time, the input path is not only used to get record reader, but
> also used for hive to get aliases and thus operator tree. But the
> CombineHiveInputFormat can have multiple paths for each split, and when
> switching paths, it also set the job with new input file name. So it always
> require a real input path name. Can not fake it.
> 2) if write a new input format, it will require a lot of duplication work
> with existing CombineHiveInputFormat.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
