[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454804#comment-16454804 ] Eugene Koifman commented on HIVE-19124: --- +1 > implement a basic major compactor for

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454745#comment-16454745 ] Sergey Shelukhin commented on HIVE-19124: - [~gopalv] now that the write ID stuff h

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453296#comment-16453296 ] Hive QA commented on HIVE-19124: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453257#comment-16453257 ] Hive QA commented on HIVE-19124: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || S

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452891#comment-16452891 ] Eugene Koifman commented on HIVE-19124: --- LGTM. Thank you. > implement a basic majo

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451457#comment-16451457 ] Sergey Shelukhin commented on HIVE-19124: - Updated > implement a basic major comp

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451283#comment-16451283 ] Eugene Koifman commented on HIVE-19124: --- My point was that instead of letting the Dr

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451281#comment-16451281 ] Sergey Shelukhin commented on HIVE-19124: - Updated the patch. I ended up splitting

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451247#comment-16451247 ] Sergey Shelukhin commented on HIVE-19124: - The patch currently extracts the write

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451231#comment-16451231 ] Eugene Koifman commented on HIVE-19124: --- a simpler approach would have been to addre

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450409#comment-16450409 ] Gopal V commented on HIVE-19124: bq. I'll take a look into the root canal thru the ear var

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450393#comment-16450393 ] Sergey Shelukhin commented on HIVE-19124: - Frankly having to do this points at the

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450388#comment-16450388 ] Gopal V commented on HIVE-19124: You need to modify the TxnUtils to accept a ValidReaderWr

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450387#comment-16450387 ] Sergey Shelukhin commented on HIVE-19124: - There's tons of code in AcidUtils that

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450383#comment-16450383 ] Eugene Koifman commented on HIVE-19124: --- I saw the comment. That isn't acceptable.

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450379#comment-16450379 ] Gopal V commented on HIVE-19124: Yes, this is true even today, but it will move away with

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450380#comment-16450380 ] Sergey Shelukhin commented on HIVE-19124: - See the comment on RB; we don't have th

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450373#comment-16450373 ] Eugene Koifman commented on HIVE-19124: --- Hold on. We already have a mechanism for c

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450365#comment-16450365 ] Sergey Shelukhin commented on HIVE-19124: - Well, it already needs to have access t

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450358#comment-16450358 ] Gopal V commented on HIVE-19124: LGTM - +1 This ticket opens up an interesting question

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449051#comment-16449051 ] Hive QA commented on HIVE-19124: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449002#comment-16449002 ] Hive QA commented on HIVE-19124: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || S

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448891#comment-16448891 ] Hive QA commented on HIVE-19124: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448735#comment-16448735 ] Sergey Shelukhin commented on HIVE-19124: - Cannot repro the only seemingly relevan

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448714#comment-16448714 ] Sergey Shelukhin commented on HIVE-19124: - Addressed the recent CR feedback > imp

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447081#comment-16447081 ] Hive QA commented on HIVE-19124: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447073#comment-16447073 ] Hive QA commented on HIVE-19124: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || S

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446493#comment-16446493 ] Sergey Shelukhin commented on HIVE-19124: - Removed the parts of the test that test

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446486#comment-16446486 ] Sergey Shelukhin commented on HIVE-19124: - Heh, the only relevant failures are IOW

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446450#comment-16446450 ] Hive QA commented on HIVE-19124: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446395#comment-16446395 ] Hive QA commented on HIVE-19124: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || S

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444929#comment-16444929 ] Eugene Koifman commented on HIVE-19124: --- left some RB comments (btw, it seems to hav

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444680#comment-16444680 ] Sergey Shelukhin commented on HIVE-19124: - Again for HiveQA > implement a basic m

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441749#comment-16441749 ] Sergey Shelukhin commented on HIVE-19124: - Addressing table type and other charact

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441693#comment-16441693 ] Hive QA commented on HIVE-19124: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441663#comment-16441663 ] Hive QA commented on HIVE-19124: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || S

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440125#comment-16440125 ] Sergey Shelukhin commented on HIVE-19124: - [~ekoifman] [~gopalv] can you please re

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440123#comment-16440123 ] Sergey Shelukhin commented on HIVE-19124: - Fixed some issues and added a test. >

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429451#comment-16429451 ] Hive QA commented on HIVE-19124: Here are the results of testing the latest attachment:

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429412#comment-16429412 ] Hive QA commented on HIVE-19124: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || S

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429113#comment-16429113 ] Sergey Shelukhin commented on HIVE-19124: - RB doesn't open, so no RB update. > im

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429108#comment-16429108 ] Sergey Shelukhin commented on HIVE-19124: - WIP patch based on 01 to address the wa

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429094#comment-16429094 ] Sergey Shelukhin commented on HIVE-19124: - The issue for compactor specifically co

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429030#comment-16429030 ] Sergey Shelukhin commented on HIVE-19124: - Fixing initiator to not skip MM tables,

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16428885#comment-16428885 ] Sergey Shelukhin commented on HIVE-19124: - The last point is actually a bug with i

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16428599#comment-16428599 ] Eugene Koifman commented on HIVE-19124: --- The flow for Acid compaction is 1. Initiat

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427857#comment-16427857 ] Sergey Shelukhin commented on HIVE-19124: - [~ekoifman] can you take a look? thnx

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427853#comment-16427853 ] Sergey Shelukhin commented on HIVE-19124: - Probably also needs a config setting to

[jira] [Commented] (HIVE-19124) implement a basic major compactor for MM tables

[ https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427849#comment-16427849 ] Sergey Shelukhin commented on HIVE-19124: - A query based compactor with tests. Pro