[
https://issues.apache.org/jira/browse/HIVE-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631241#comment-13631241
]
Hudson commented on HIVE-4322:
------------------------------
Integrated in Hive-trunk-hadoop2 #157 (See
[https://builds.apache.org/job/Hive-trunk-hadoop2/157/])
HIVE-4322: SkewedInfo in Metastore Thrift API cannot be deserialized in
Python (Samuel Yuan via Gang Tim Liu) (Revision 1467491)
Result = FAILURE
gangtimliu :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1467491
Files :
* /hive/trunk/metastore/if/hive_metastore.thrift
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/EnvironmentContext.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrincipalPrivilegeSet.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SerDeInfo.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
*
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
* /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php
* /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
* /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
*
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
*
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPruner.java
*
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ListBucketingCtx.java
> SkewedInfo in Metastore Thrift API cannot be deserialized in Python
> -------------------------------------------------------------------
>
> Key: HIVE-4322
> URL: https://issues.apache.org/jira/browse/HIVE-4322
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Thrift API
> Affects Versions: 0.11.0
> Reporter: Samuel Yuan
> Assignee: Samuel Yuan
> Priority: Minor
> Fix For: 0.11.0
>
> Attachments: HIVE-4322.HIVE-4322.HIVE-4322.HIVE-4322.D10203.1.patch
>
>
> The Thrift-generated Python code that deserializes Thrift objects fails
> whenever a complex type is used as a map key, because by default mutable
> Python objects such as lists do not have a hash function. See
> https://issues.apache.org/jira/browse/THRIFT-162 for related discussion.
> The SkewedInfo struct contains a map which uses a list as a key, breaking the
> Python Thrift interface. It is not possible to specify the mapping from
> Thrift types to Python types, or otherwise we could map Thrift lists to
> Python tuples. Instead, the proposed workaround wraps the list inside a new
> struct. This alone does not accomplish anything, but allows Python clients to
> define a hash function for the struct class, e.g.:
> def f(object):
> return hash(tuple(object.skewedValueList))
> SkewedValueList.__hash__ = f
> In practice a more efficient hash might be defined that does not involve
> copying the list. The advantage of wrapping the list inside a struct is that
> the client does not have to define the hash on the list itself, which would
> change the behaviour of lists everywhere else in the code.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
