[
https://issues.apache.org/jira/browse/HIVE-24693?focusedWorklogId=544637&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-544637
]
ASF GitHub Bot logged work on HIVE-24693:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Jan/21 02:29
Start Date: 30/Jan/21 02:29
Worklog Time Spent: 10m
Work Description: belugabehr closed pull request #1918:
URL: https://github.com/apache/hive/pull/1918
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 544637)
Time Spent: 20m (was: 10m)
> Parquet Timestamp Values Read/Write Very Slow
> ---------------------------------------------
>
> Key: HIVE-24693
> URL: https://issues.apache.org/jira/browse/HIVE-24693
> Project: Hive
> Issue Type: Improvement
> Reporter: David Mollitor
> Assignee: David Mollitor
> Priority: Critical
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Parquet {{DataWriteableWriter}} relias on {{NanoTimeUtils}} to convert a
> timestamp object into a binary value. The way in which it does this,... it
> calls {{toString()}} on the timestamp object, and then parses the String.
> This particular timestamp do not carry a timezone, so the string is something
> like:
> {{2021-21-03 12:32:23.0000...}}
> The parse code tries to parse the string assuming there is a time zone, and
> if not, falls-back and applies the provided "default time zone". As was
> noted in [HIVE-24353], if something fails to parse, it is very expensive to
> try to parse again. So, for each timestamp in the Parquet file, it:
> * Builds a string from the time stamp
> * Parses it (throws an exception, parses again)
> There is no need to do this kind of string manipulations/parsing, it should
> just be using the epoch millis/seconds/time stored internal to the Timestamp
> object.
> {code:java}
> // Converts Timestamp to TimestampTZ.
> public static TimestampTZ convert(Timestamp ts, ZoneId defaultTimeZone) {
> return parse(ts.toString(), defaultTimeZone);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
