Hi all:
I try to run simple transform script in hive and my script is written in
python. But when I try to import other file in the script the task fails.
There may be some basic classes which are used by many transform scripts,
so I would like to know how to import other file in my transform script.
Below is my test data:

Hive CLI Query:
add file /home/hive_test/empty.py;
add file /home/hive_test/normal.py;
insert overwrite table test
select
  transform(d.*)
     using './normal.py'
     as (`name`)
from
(
   select
     name
   from test_1
) d;

normal.py:
#!/usr/bin/python

import sys
import empty

if __name__ == "__main__":
        for l in sys.stdin:
                print l.strip('\n')

empty.py:
#!/usr/bin/python
class Empty:
        pass



error msg:

java.lang.RuntimeException: Hive Runtime Error while closing operators
        at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:226)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hit error
while closing ..
        at 
org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
        at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193)
        ... 8 more


Thanks.

Reply via email to