[
https://issues.apache.org/jira/browse/HIVE-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-2820:
-----------------------------------
Affects Version/s: 0.10.0
Status: Open (was: Patch Available)
I was able to resolve conflict and apply the patch. Than ran the tests. All the
tests passed except TestCliDriver_join_reorder2.q which failed with following
stack:
{code}
[junit] java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
[junit] at java.util.ArrayList.RangeCheck(ArrayList.java:547)
[junit] at java.util.ArrayList.get(ArrayList.java:322)
[junit] at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:118)
[junit] at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.<init>(StandardStructObjectInspector.java:106)
[junit] at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:280)
[junit] at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:265)
[junit] at
org.apache.hadoop.hive.ql.exec.JoinUtil.getRowContainer(JoinUtil.java:405)
[junit] at
org.apache.hadoop.hive.ql.exec.AbstractMapJoinOperator.initializeOp(AbstractMapJoinOperator.java:101)
[junit] at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.initializeOp(MapJoinOperator.java:80)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:436)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:392)
[junit] at
org.apache.hadoop.hive.ql.exec.HashTableDummyOperator.initializeOp(HashTableDummyOperator.java:39)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
[junit] at
org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:110)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
[junit] at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
[junit] at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
[junit] at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
[junit] at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
[junit] at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
[junit] at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
[junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
[junit] at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
[junit] java.lang.NullPointerException
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:530)
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:859)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:558)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:200)
[junit] at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
[junit] at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
[junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
[junit] at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
[junit] org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:536)
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:859)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:558)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:200)
[junit] at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
[junit] at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
[junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
[junit] at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
[junit] Caused by: java.lang.NullPointerException
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:530)
[junit] ... 10 more
[junit] org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:536)
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:859)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:558)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:200)
[junit] at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
[junit] at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
[junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
[junit] at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
[junit] Caused by: java.lang.NullPointerException
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:530)
[junit] ... 10 more
[junit] org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:536)
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:859)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:558)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:567)
[junit] at
org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:200)
[junit] at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
[junit] at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
[junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
[junit] at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
[junit] Caused by: java.lang.NullPointerException
[junit] at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:530)
[junit] ... 10 more
[junit] Ended Job = job_local_0001 with errors
{code}
Its possible that I didn't resolve conflicts correctly.
> Invalid tag is used for MapJoinProcessor
> ----------------------------------------
>
> Key: HIVE-2820
> URL: https://issues.apache.org/jira/browse/HIVE-2820
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.9.0, 0.10.0
> Environment: ubuntu
> Reporter: Navis
> Assignee: Navis
> Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2820.D1935.1.patch,
> ASF.LICENSE.NOT.GRANTED--HIVE-2820.D1935.2.patch,
> ASF.LICENSE.NOT.GRANTED--HIVE-2820.D1935.3.patch
>
>
> Testing HIVE-2810, I've found tag and alias are used in very confusing
> manner. For example, query below fails..
> {code}
> hive> set hive.auto.convert.join=true;
>
> hive> select /*+ STREAMTABLE(a) */ * from myinput1 a join myinput1 b on
> a.key=b.key join myinput1 c on a.key=c.key;
> Total MapReduce jobs = 4
> Ended Job = 1667415037, job is filtered out (removed at runtime).
> Ended Job = 1739566906, job is filtered out (removed at runtime).
> Ended Job = 1113337780, job is filtered out (removed at runtime).
> 12/02/24 10:27:14 WARN conf.HiveConf: DEPRECATED: Ignoring hive-default.xml
> found on the CLASSPATH at /home/navis/hive/conf/hive-default.xml
> Execution log at:
> /tmp/navis/navis_20120224102727_cafe0d8d-9b21-441d-bd4e-b83303b31cdc.log
> 2012-02-24 10:27:14 Starting to launch local task to process map join;
> maximum memory = 932118528
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.processOp(HashTableSinkOperator.java:312)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at
> org.apache.hadoop.hive.ql.exec.MapredLocalTask.startForward(MapredLocalTask.java:325)
> at
> org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:272)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:685)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> Execution failed with exit status: 2
> Obtaining error information
> {code}
> Failed task has a plan which doesn't make sense.
> {noformat}
> Stage: Stage-8
> Map Reduce Local Work
> Alias -> Map Local Tables:
> b
> Fetch Operator
> limit: -1
> c
> Fetch Operator
> limit: -1
> Alias -> Map Local Operator Tree:
> b
> TableScan
> alias: b
> HashTable Sink Operator
> condition expressions:
> 0 {key} {value}
> 1 {key} {value}
> 2 {key} {value}
> handleSkewJoin: false
> keys:
> 0 [Column[key]]
> 1 [Column[key]]
> 2 [Column[key]]
> Position of Big Table: 0
> c
> TableScan
> alias: c
> Map Join Operator
> condition map:
> Inner Join 0 to 1
> Inner Join 0 to 2
> condition expressions:
> 0 {key} {value}
> 1 {key} {value}
> 2 {key} {value}
> handleSkewJoin: false
> keys:
> 0 [Column[key]]
> 1 [Column[key]]
> 2 [Column[key]]
> outputColumnNames: _col0, _col1, _col4, _col5, _col8, _col9
> Position of Big Table: 0
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> expr: _col4
> type: int
> expr: _col5
> type: int
> expr: _col8
> type: int
> expr: _col9
> type: int
> outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-4
> Map Reduce
> Alias -> Map Operator Tree:
> a
> TableScan
> alias: a
> HashTable Sink Operator
> condition expressions:
> 0 {key} {value}
> 1 {key} {value}
> 2 {key} {value}
> handleSkewJoin: false
> keys:
> 0 [Column[key]]
> 1 [Column[key]]
> 2 [Column[key]]
> Position of Big Table: 0
> Local Work:
> Map Reduce Local Work
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
