[
https://issues.apache.org/jira/browse/HIVE-24688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-24688:
--------------------------------
Description:
It's not necessarily copyToStandardObject which should be optimized, but we
need to consider some optimization on the attached codepath.
In a customer case, 3 reducer tasks run forever (handling skewed keys) and most
of the time is spent on this code path, utilizing GC heavily. At the moment I'm
open to any kind of optimization:
- 1. do we need to copy Text? cannot we get a reference back?-
2. Maintain a writable cache with keys as Writable.hashCode from RowContainer
!Screen Shot 2021-01-27 at 9.52.32 AM.png|width=652,height=280!
was:
It's not necessarily copyToStandardObject which should be optimized, but we
need to consider some optimization on the attached codepath.
In a customer case, 3 reducer tasks run forever (handling skewed keys) and most
of the time is spent on this code path, utilizing GC heavily. At the moment I'm
open to any kind of optimization:
1. do we need to copy Text? cannot we get a reference back?
!Screen Shot 2021-01-27 at 9.52.32 AM.png|width=652,height=280!
> Optimise ObjectInspectorUtils.copyToStandardObject
> --------------------------------------------------
>
> Key: HIVE-24688
> URL: https://issues.apache.org/jira/browse/HIVE-24688
> Project: Hive
> Issue Type: Improvement
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Attachments: Screen Shot 2021-01-27 at 9.52.32 AM.png
>
>
> It's not necessarily copyToStandardObject which should be optimized, but we
> need to consider some optimization on the attached codepath.
> In a customer case, 3 reducer tasks run forever (handling skewed keys) and
> most of the time is spent on this code path, utilizing GC heavily. At the
> moment I'm open to any kind of optimization:
> - 1. do we need to copy Text? cannot we get a reference back?-
> 2. Maintain a writable cache with keys as Writable.hashCode from RowContainer
> !Screen Shot 2021-01-27 at 9.52.32 AM.png|width=652,height=280!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
