[
https://issues.apache.org/jira/browse/HIVE-27965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naresh P R updated HIVE-27965:
------------------------------
Description:
Partition table rename gets clogged at PART_COL_STATS for wide tables.
{code:java}
CREATE TABLE IF NOT EXISTS `PART_COL_STATS` (
...
`DB_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
`TABLE_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
`PARTITION_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT
NULL,
...){code}
Since PART_COL_STATS holds db_name & table_name, incase of table rename, every
row in PART_COL_STATS associated with the table should be fetched, stored in
memory, delete & re-insert with new db/table/partition name.
Remove DB_NAME, TABLE_NAME, PARTITION_NAME from PART_COL_STATS & use & use
TBL_ID, DB_ID, PART_ID to avoid touching PART_COL_STATS for table/partition
renames.
Also TBL_ID, DB_ID, PART_ID can be used for PART_COL_STATS INDEXING.
was:
Partition table rename gets clogged at PART_COL_STATS for wide tables.
{code:java}
CREATE TABLE IF NOT EXISTS `PART_COL_STATS` (
...
`DB_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
`TABLE_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
`PARTITION_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT
NULL,
...){code}
Since PART_COL_STATS holds db_name & table_name, incase of table rename, every
row in PART_COL_STATS associated with the table should be fetched, stored in
memory, delete & re-insert with new db/table/partition name.
Remove DB_NAME, TABLE_NAME, PARTITION_NAME from PART_COL_STATS & use PART_ID as
FOREIGN KEY from PARTITIONS to avoid touching PART_COL_STATS for
table/partition renames.
> Table/partition rename takes a long time at PART_COL_STATS for wide tables
> --------------------------------------------------------------------------
>
> Key: HIVE-27965
> URL: https://issues.apache.org/jira/browse/HIVE-27965
> Project: Hive
> Issue Type: Improvement
> Reporter: Naresh P R
> Priority: Major
>
> Partition table rename gets clogged at PART_COL_STATS for wide tables.
> {code:java}
> CREATE TABLE IF NOT EXISTS `PART_COL_STATS` (
> ...
> `DB_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
> `TABLE_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
> `PARTITION_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT
> NULL,
> ...){code}
> Since PART_COL_STATS holds db_name & table_name, incase of table rename,
> every row in PART_COL_STATS associated with the table should be fetched,
> stored in memory, delete & re-insert with new db/table/partition name.
>
> Remove DB_NAME, TABLE_NAME, PARTITION_NAME from PART_COL_STATS & use & use
> TBL_ID, DB_ID, PART_ID to avoid touching PART_COL_STATS for table/partition
> renames.
> Also TBL_ID, DB_ID, PART_ID can be used for PART_COL_STATS INDEXING.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
