Hi all: I try to run simple transform script in hive and my script is written in python. But when I try to import other file in the script the task fails. There may be some basic classes which are used by many transform scripts, so I would like to know how to import other file in my transform script. Below is my test data:
Hive CLI Query: add file /home/hive_test/empty.py; add file /home/hive_test/normal.py; insert overwrite table test select transform(d.*) using './normal.py' as (`name`) from ( select name from test_1 ) d; normal.py: #!/usr/bin/python import sys import empty if __name__ == "__main__": for l in sys.stdin: print l.strip('\n') empty.py: #!/usr/bin/python class Empty: pass error msg: java.lang.RuntimeException: Hive Runtime Error while closing operators at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:226) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193) ... 8 more Thanks.