[
https://issues.apache.org/jira/browse/HIVE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569507#comment-14569507
]
prashant kumar commented on HIVE-5795:
--------------------------------------
Hi, I am trying to use below properties in a hive table .
'skip.footer.line.count'='1',
'skip.header.line.count'='1'
These have been added as an alter table and not at the time of table creation.
And I am trying to use a fixed width ascii file.
I have used the put command to copy the file in hdfs and when i am trying to
see the data through hive, the last trailer record is also coming. But record
count is reduced by 2 in select count(*) of hive table, so it's removing some
other data record instead of trailer.
My file structure is -
Header
... data ...
... data ...
... data ...
Trailer
I have verified my data file, last record is the trailer record only and there
is no other invalid "\n" character in my file. But due to some issue, this
property is removing some other data record.
Did anyone else has faced this issue ? OR Can someone please help me to
identify what's wrong here ?
hive --version
Hive 0.13.1-cdh5.3.2
> Hive should be able to skip header and footer rows when reading data file for
> a table
> -------------------------------------------------------------------------------------
>
> Key: HIVE-5795
> URL: https://issues.apache.org/jira/browse/HIVE-5795
> Project: Hive
> Issue Type: New Feature
> Reporter: Shuaishuai Nie
> Assignee: Shuaishuai Nie
> Labels: TODOC13
> Fix For: 0.13.0
>
> Attachments: HIVE-5795.1.patch, HIVE-5795.2.patch, HIVE-5795.3.patch,
> HIVE-5795.4.patch, HIVE-5795.5.patch
>
>
> Hive should be able to skip header and footer lines when reading data file
> from table. In this way, user don't need to processing data which generated
> by other application with a header or footer and directly use the file for
> table operations.
> To implement this, the idea is adding new properties in table descriptions to
> define the number of lines in header and footer and skip them when reading
> the record from record reader. An DDL example for creating a table with
> header and footer should be like this:
> {code}
> Create external table testtable (name string, message string) row format
> delimited fields terminated by '\t' lines terminated by '\n' location
> '/testtable' tblproperties ("skip.header.line.count"="1",
> "skip.footer.line.count"="2");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
