[
https://issues.apache.org/jira/browse/HIVE-24503?focusedWorklogId=523725&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-523725
]
ASF GitHub Bot logged work on HIVE-24503:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Dec/20 07:16
Start Date: 14/Dec/20 07:16
Worklog Time Spent: 10m
Work Description: maheshk114 commented on a change in pull request #1753:
URL: https://github.com/apache/hive/pull/1753#discussion_r542158896
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSerializeRow.java
##########
@@ -328,6 +345,13 @@ private void serializeUnionWrite(
serializeWrite.finishUnion();
}
+ class VectorSerializeStructWriter extends VectorSerializeWriter {
+ @Override
+ void serialize(Object colInfo, Field field, int adjustedBatchIndex) throws
IOException {
+ serializeStructWrite((StructColumnVector)colInfo, field,
adjustedBatchIndex);
+ }
+ }
+
private void serializeStructWrite(
Review comment:
done
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSerializeRow.java
##########
@@ -355,6 +379,13 @@ private void serializeStructWrite(
serializeWrite.finishStruct();
}
+ class VectorSerializeMapWriter extends VectorSerializeWriter {
+ @Override
+ void serialize(Object colInfo, Field field, int adjustedBatchIndex) throws
IOException {
+ serializeMapWrite((MapColumnVector)colInfo, field, adjustedBatchIndex);
+ }
+ }
+
private void serializeMapWrite(
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 523725)
Time Spent: 40m (was: 0.5h)
> Optimize vector row serde by avoiding type check at run time
> -------------------------------------------------------------
>
> Key: HIVE-24503
> URL: https://issues.apache.org/jira/browse/HIVE-24503
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Serialization/Deserialization of vectorized batch done at VectorSerializeRow
> and VectorDeserializeRow does a type checking for each column of each row.
> This becomes very costly when there are billions of rows to read/write. This
> can be optimized if the type check is done during init time and specific
> reader/writer classes are created. This classes can be used directly stored
> in filed structure to avoid run time type check.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
