I have a problem that inserting data from external table to managed table.
I already have done just 'limit 10' with below hql but can't all data.
I have the error just like '# of failed Map Tasks exceeded allowed limit'
How should I do?
---------------------------------------------------------------------------------
cause: SQLException:Error while executing statement:
INSERT OVERWRITE TABLE newTbl PARTITION(etl_ymd, etl_hh)
SELECT rec_type,
hash(substr(svd_name,1,case when instr(svd_name,'F')=0 then
length(svd_name) when instr(svd_name,'F')>30 then 30 else
instr(svd_name,'F')-1 end )) as imsi,
hash(substr(svd_nick,1,case when instr(svd_nick,'F')=0 then
length(svd_nick) when instr(svd_nick,'F')>0 then 30 else
instr(svd_nick,'F')-1 end)) as imei,
hash(a.private) as private,
hash(case when substr(a.private,3,2)='82' then
concat('0',substr(a.private,5)) else substr(a.private,5) end) as num,
evt_timestamp,
msg_ref,
hash(dest_no) as
dest_no,
system_type,
a.file_name as
file_name,
concat(substr(a.file_name,2,2),substr(a.file_name,6,1)) as own_no,
etl_ymd,
etl_hh
FROM
(
select *,
substr(svd_private,1,case when instr(svd_private,'F')=0 then
length(svd_private) when instr(svd_private,'F')>30 then 30 else
instr(svd_private,'F')-1 end) as private,
split(split(INPUT__FILE__NAME, "etl_hh=[0-9]+/")[1], ".DAT")[0] as
file_name
from oldTbl
) a
by FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask, SQLState = 08S01, ResponseCode =
2
org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:168)
org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:156)
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:198)
com.nexr.ndap.driver.hive.jdbc.HiveJdbcConnector.doExecute(HiveJdbcConnector.java:169)
com.nexr.ndap.driver.hive.jdbc.HiveJdbcConnector.executeQueryWithSession(HiveJdbcConnector.java:856)
com.nexr.ndap.driver.hive.ConcurrentHiveConnector.executeQueryWithSession(ConcurrentHiveConnector.java:283)
com.nexr.ndap.service.workbench.impl.HiveQueryServiceImpl.runQuery(HiveQueryServiceImpl.java:582)
com.nexr.ndap.service.workbench.impl.HiveQueryServiceImpl.runQuery(HiveQueryServiceImpl.java:433)
sun.reflect.GeneratedMethodAccessor743.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:80)
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
java.util.concurrent.FutureTask.run(FutureTask.java:138)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
java.lang.Thread.run(Thread.java:662)
--------------------------------------------------------------------------------
= job detail log
Hadoop job_201406192109_16353 on p-bigai-pk1-a03
User: admin
Job Name: INSERT OVERWRITE TABLE mzn.mzn_cdr_3gsms...a(Stage-1)
Job File:
hdfs://p-bigai-pk1-a03:9000/tmp/hadoop-mapred/mapred/staging/admin/.staging/job_201406192109_16353/job.xml
Submit Host: p-bigai-pk1-a01
Submit Host Address: 10.220.78.91
Job-ACLs: All users are allowed
Job Setup: Successful
Status: Failed
Failure Info:# of failed Map Tasks exceeded allowed limit. FailedCount: 1.
LastFailedTask: task_201406192109_16353_m_000006
Started at: Wed Dec 03 14:18:36 KST 2014
Failed at: Wed Dec 03 14:19:04 KST 2014
Failed in: 27sec
Job Cleanup: Successful
Black-listed TaskTrackers: 19
