[
https://issues.apache.org/jira/browse/HIVE-27991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17950021#comment-17950021
]
Denys Kuzmenko commented on HIVE-27991:
---------------------------------------
hi [~sbadhya] , [~zhangbutao] what this functionality ever used? For the
partitioned iceberg tables, we always add a
customSortExpr per partition spec.
I am looking at HIVE-25948 and planning to incorporate
HIVE_OPT_SORT_DYNAMIC_PARTITION_THRESHOLD to decide if we need any dyn sort
expressions.
> Utilise FanoutWriters when inserting records in an Iceberg table when the
> records are unsorted
> ----------------------------------------------------------------------------------------------
>
> Key: HIVE-27991
> URL: https://issues.apache.org/jira/browse/HIVE-27991
> Project: Hive
> Issue Type: Improvement
> Reporter: Sourabh Badhya
> Assignee: Sourabh Badhya
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> FanoutWriter is a writer in Iceberg which can be used to write records in a
> table. This writer keeps all the file handles open, until the write is
> finished. FanoutWriters is used as the writer when the incoming records are
> unsorted. We can by default have some mechanism to switch to using
> FanoutWriters instead of ClusteredWriters when custom sort expressions are
> not present for the given table/query.
> Similar stuff is already implemented in Spark -
> [https://github.com/apache/iceberg/pull/8621]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
