[
https://issues.apache.org/jira/browse/HIVE-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371746#comment-15371746
]
Owen O'Malley edited comment on HIVE-13974 at 7/11/16 10:09 PM:
----------------------------------------------------------------
{quote}
No, the semantics of sameCategoryAndAttributes is different than equals.
{quote}
*Sigh* Ok, I forgot that I had only fixed that on the ORC side of the world as
part of ORC-53. Hive will get that as soon as HIVE-14007 goes in (or is a
negative patch of 2MB "going out"?). In any case, do not add the new method.
ORC-53's impact on orc-core is pretty small outside of TypeDescription. Would
you like a back port of that patch?
{quote}
There are 3 kinds of schema not 2.
{quote}
Ugh. That seems unnecessary. The 'file' schema is pretty clear. The 'reader'
schema is the one that the user asked for. I don't think we need anything else.
{quote}
About ORC-54 -- it is not practical right now in terms of time.
{quote}
ORC-54 is closer to going in. It has unit tests and I believe handles this as a
sub-case. I'm trying to figure out what we gain out of the HIVE-13974 patch.
{quote}
Also, there really needs to be a parallel HIVE JIRA for it and we must make
sure name mapping is fully supported for
{quote}
Uh no. The Hive ORC code is about to disappear with HIVE-14007. Continuing to
maintain two versions of ORC with a forked code base is a bad thing.
{quote}
Given how *difficult* Schema Evolution has been I simply don't believe it will
*just work* with ORC only unit tests.
{quote}
That is not an excuse. Unit tests are MUCH more likely to be correct because
the errors aren't hidden under layers of the execution engine. Being difficult
to get right is why not having unit tests is unacceptable.
was (Author: owen.omalley):
{quote}
No, the semantics of sameCategoryAndAttributes is different than equals.
{quote}
*Sigh* Ok, I forgot that I had only fixed that on the ORC side of the world as
part of ORC-53. Hive will get that as soon as HIVE-14007 goes in (or is a
negative patch of 2MB "going out"?). In any case, do not add the new method.
ORC-53's impact on orc-core is pretty small outside of TypeDescription. Would
you like a back port of that patch?
{quote}
There are 3 kinds of schema not 2.
{quote}
Ugh. That seems unnecessary. The 'file' schema is pretty clear. The 'reader'
schema is the one that the user asked for. I don't think we need anything else.
{quote}
About ORC-54 -- it is not practical right now in terms of time.
{quote}
ORC-54 is closer to going in. It has unit tests and I believe handles this as a
sub-case. I'm trying to figure out what we gain out of the HIVE-13974 patch.
{quote}
Also, there really needs to be a parallel HIVE JIRA for it and we must make
sure name mapping is fully supported for
Uh no. The Hive ORC code is about to disappear with HIVE-14007. Continuing to
maintain two versions of ORC with a forked code base is a bad thing.
{quote}
Given how *difficult* Schema Evolution has been I simply don't believe it will
*just work* with ORC only unit tests.
{quote}
That is not an excuse. Unit tests are MUCH more likely to be correct because
the errors aren't hidden under layers of the execution engine.
> ORC Schema Evolution doesn't support add columns to non-last STRUCT columns
> ---------------------------------------------------------------------------
>
> Key: HIVE-13974
> URL: https://issues.apache.org/jira/browse/HIVE-13974
> Project: Hive
> Issue Type: Bug
> Components: Hive, ORC, Transactions
> Affects Versions: 1.3.0, 2.1.0, 2.2.0
> Reporter: Matt McCline
> Assignee: Matt McCline
> Priority: Blocker
> Attachments: HIVE-13974.01.patch, HIVE-13974.02.patch,
> HIVE-13974.03.patch, HIVE-13974.04.patch, HIVE-13974.05.WIP.patch,
> HIVE-13974.06.patch, HIVE-13974.07.patch, HIVE-13974.08.patch,
> HIVE-13974.09.patch, HIVE-13974.091.patch
>
>
> Currently, the included columns are based on the fileSchema and not the
> readerSchema which doesn't work for adding columns to non-last STRUCT data
> type columns.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
