[
https://issues.apache.org/jira/browse/HIVE-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029057#comment-13029057
]
Vishal Kathuria commented on HIVE-2100:
---------------------------------------
This patch contains the fix for the issue as well as the tests for it.
The problem was that the object inspector was initialized and added to the
opCtxMap before the check was done for which table this operation belongs to.
For the object with virtual columns, fixing up the inspector for virtual
columns happens after this check. So we were left with an inspector in the
ctxMap that didn't account for the virtual columns and errored out during the
initialization.
With this change, the extra inspector wouldn't get unnecessarily created.
> virtual column references inside subqueries cause execution exceptions
> ----------------------------------------------------------------------
>
> Key: HIVE-2100
> URL: https://issues.apache.org/jira/browse/HIVE-2100
> Project: Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Attachments: HIVE-2100.txt
>
>
> example:
> create table jssarma_nilzma_bad as select a.fname, a.offset, a.val from
> (select
> hash(eventid,userid,eventtime,browsercookie,userstate,useragent,userip,serverip,clienttime,geoid,countrycode\
> ,actionid,lastimpressionid,lastnavimpressionid,impressiontype,fullurl,fullreferrer,pagesection,modulesection,adsection)
> as val, INPUT__FILE__NAME as fname, BLOCK__OFFSET__INSIDE__FILE as offset
> from nectar_impression_lzma_unverified where ds='2010-07-28') a join
> jssarma_hc_diff b on (a.val=b.val);"
> causes
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: cannot find field input__file__name
> from
> [org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@664310d0,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@3d04fc23,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@12457d21,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@101a0ae6,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@1dc18a4c,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@d5e92d7,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@3bfa681c,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@34c92507,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@19e09a4,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@2e8aeed0,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@2344b18f,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@72e5355f,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@26132ae7,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@3465b738,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@1dfd868,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@ef894ce,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@61f1680f,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@2fe6e305,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@5f4275d4,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@445e228,
>
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@802b249]
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:321)
> at
> org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector.getStructFieldRef(UnionStructObjectInspector.java:96)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at
> org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:73)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at
> org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> running the subquery separately fixes the issue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
