GitHub user ashutosh-bapat opened a pull request:
https://github.com/apache/hive/pull/522
Hive21079: Stats replication for partitioned table
The first commit is for stats replication for partitioned table. The other
two commits are fixing bugs in existing code, AFAIU.
@sankarh can you please review?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ashutosh-bapat/hive hive21079
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hive/pull/522.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #522
----
commit a8b729ab1f120cd50c0ad0e096bb0f724a178838
Author: Ashutosh Bapat <ashutosh.bapat.oss@...>
Date: 2019-01-15T12:06:29Z
HIVE-21079: Replicate statistics for partitioned, non-transactional tables.
Ashutosh Bapat
commit c8aea7b85a06ab53873ce60eb08fcf0514806787
Author: Ashutosh Bapat <ashutosh.bapat.oss@...>
Date: 2019-01-18T05:31:44Z
HIVE-21079: ALTER PARTITION events not applied during incremental
replication
In AlterPartitionHandler, we set
withinContext.replicationSpec.setIsMetadataOnly(true);
In ImportSemanticAnalyzer.createReplImportTasks(), per code around line
1197, we do not add new
PartitionSpecs and corresponding tasks. This means that we never apply an
ALTER_PARTITION event
during incremental load. That looks like a serious bug.
Either we should check PartitionDescs irrespective of
replicationSpec.setIsMetadataOnly() OR we
shouldnât set replicationSpec.setIsMetadataOnly() to true while dumping
an ALTER_PARTITION event. We
set replicationSpec.setIsMetadataOnly(true) for ALTER TABLE events as well,
so doing that for ALTER
PARTITION event looks fine.
Ashutosh Bapat.
commit 536492395cd5c280738c2ec1038c39036b477209
Author: Ashutosh Bapat <ashutosh.bapat.oss@...>
Date: 2019-01-18T06:07:37Z
HIVE-21079: Do not dump partition related events during a metadata only
dump.
During bootstrap metadata-only dump we do not dump partitions (See
TableExport.getPartitions(). For
bootstrap dump we always pass TableSpec with TABLE_ONLY set.). So don't
dump partition related
events for a metadata-only dump.
Ashutosh Bapat.
----
---
