[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16240887#comment-16240887
]
Sergey Shelukhin commented on HIVE-17856:
-----------------------------------------
Is it possible to post a review board review
(https://reviews.apache.org/r/new/, project hive-git) or a pull request
(whichever you prefer)?
I have some minor comments that I can post there. The ones though,
1) It should be possible to remove the code that deletes "other" deltas with
this patch, and probably to remove "isMatch" argument from IdPathFilter, since
it was only used for that.
2) On the read side, where it adds the base (" // Find the base, created for
IOW."), shouldn't it also do something to skip the deltas before the last
committed base? Maybe there's logic in ACID somewhere; it may be hard to reuse
directly given a different layout, but perhaps it can be duplicated or modified
cc [~ekoifman]
3) Tests appear to have failed (mm_all).
4) getValidPartitionsInPath passes in false, although it is possible to have
dynamic partitioning with IOW.
4a) In general though, one txn can only create either base or delta in each
given location (or multiple of one type), as far as I understand. So it may be
possible to change the filter/etc. to find both by txn number, instead of
passing the flag and only finding one type. Then, we can error out if both are
present at once.
4b) Actually [~ekoifman], that makes me wonder, about IOW with union work for
ACID? ACID union insert creates two deltas; does ACID IOW with union create two
base directories?
> MM tables - IOW is not ACID compliant
> -------------------------------------
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
> Reporter: Sergey Shelukhin
> Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
