hi all:
I have found the answer here
<http://osdir.com/ml/hive-user-hadoop-apache/2010-05/msg00038.html>
:
Adding the following lines before the import
solved the problem:
import sys
import os
sys.path.append(os.getcwd())
2012/5/14 wzc <wzc1...@gmail.com>
> Hi all:
> I try to run simple transform script in hive and my script is written in
> python. But when I try to import other file in the script the task fails.
> There may be some basic classes which are used by many transform scripts,
> so I would like to know how to import other file in my transform script.
> Below is my test data:
>
> Hive CLI Query:
> add file /home/hive_test/empty.py;
> add file /home/hive_test/normal.py;
> insert overwrite table test
> select
> transform(d.*)
> using './normal.py'
> as (`name`)
> from
> (
> select
> name
> from test_1
> ) d;
>
> normal.py:
> #!/usr/bin/python
>
> import sys
> import empty
>
> if __name__ == "__main__":
> for l in sys.stdin:
> print l.strip('\n')
>
> empty.py:
> #!/usr/bin/python
> class Empty:
> pass
>
>
>
> error msg:
>
> java.lang.RuntimeException: Hive Runtime Error while closing operators
> at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:226)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while
> closing ..
> at
> org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193)
> ... 8 more
>
>
> Thanks.
>
