[
https://issues.apache.org/jira/browse/IMPALA-13738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17939232#comment-17939232
]
ASF subversion and git services commented on IMPALA-13738:
----------------------------------------------------------
Commit 0b3bc412f2b09f4818cb0c50fccec5cfb3ddfe8d in impala's branch
refs/heads/master from Noemi Pap-Takacs
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0b3bc412f ]
IMPALA-13738 (Part1): Refactor IcebergTable.getPartialInfo()
This patch aims to refactor Iceberg tables in the Catalog.
It reduces the dependency on the internal hdfsTable_ object in
IcebergTable.getPartialInfo(). Now we depend less on HdfsTable
when we load an Iceberg table in local catalog mode.
The remaining dependencies will be addressed in following patches.
Now creating TPartialPartitionInfo is the responsibility of the
partitions, not the tables:
- FeFsPartition holds a basic default implementation for the
unpartitioned tables, which includes all Iceberg tables (we
handle Iceberg tables similarly to unpartitioned HDFS tables
in the Catalog).
- HdfsPartition extends this implementation with HDFS and
Hive ACID specific fields.
Testing:
- existing Iceberg e2e tests in local catalog mode
- existing Hive ACID tests in local catalog mode
- ran test_iceberg_with_puffin.py in local catalog mode
- LocalCatalog FE test
Change-Id: I75d277fb555d0088a2e28111ff529a4605d734fa
Reviewed-on: http://gerrit.cloudera.org:8080/22605
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Extract common code of IcebergTable and HdfsTable to a helper class
> -------------------------------------------------------------------
>
> Key: IMPALA-13738
> URL: https://issues.apache.org/jira/browse/IMPALA-13738
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Zoltán Borók-Nagy
> Assignee: Noémi Pap-Takács
> Priority: Major
> Labels: impala-iceberg
>
> Currently, IcebergTable has an underlying HdfsTable that gives us headaches
> from time to time.
> We should identify the common data members and logic between IcebergTable and
> HdfsTable and put them into a helper class and both IcebergTable and
> HdfsTable would use that class internally.
> HdfsTable and IcebergTable have a common base interface FeFsTable, but we
> cannot put useful code there.
> OTOH, HdfsTable and IcebergTable both extend Table, so as an alternative
> option (to composition), it might be possible to put common code into a newly
> introduced common base class:
> {noformat}
> Table
> ^
> |
> FsTable (NEW)
> ^
> / \
> / \
> HdfsTable IcebergTable{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
