Senthil Ganesh created SQOOP-3065: ------------------------------------- Summary: Sqoop Import fails when column containing unicode data for split-by Key: SQOOP-3065 URL: https://issues.apache.org/jira/browse/SQOOP-3065 Project: Sqoop Issue Type: Bug Components: connectors/sqlserver Reporter: Senthil Ganesh
sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true --connect jdbc:sqlserver://10.65.141.234;databaseName=sqoop_adapter --password ****** --map-column-java LANGUAGE=String,CODEPAGE=String,C_NAME_JAP_名前=String,C_ADD_GREEK_ΔΙΕΎΘΥΝΣΗ=String,C_MKTSGMT_CHINESE_市場=String,C_COMMENT_KOREAN_덧글=String --username sq_admin --as-avrodatafile --split-by C_MKTSGMT_CHINESE_市場 --columns LANGUAGE,CODEPAGE,C_NAME_JAP_名前,C_ADD_GREEK_ΔΙΕΎΘΥΝΣΗ,C_MKTSGMT_CHINESE_市場,C_COMMENT_KOREAN_덧글 --table 日本語_ΕΛΛΗΝΙΚΑ --target-dir hdfs://asd/blaze/workdir/4133141082177241108/W4133141082177241108_Read_日本語_ΕΛΛΗΝΙΚΑ_m_export_日本語_ΕΛΛΗΝΙΚΑ/staged_files overrides a deprecated API. 2016-12-01 11:50:22 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Note: Recompile with -Xlint:deprecation for details. 2016-12-01 11:50:22 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Writing jar file: /tmp/sqoop-adpqa/compile/9013a77a704c396d503bc9c56a95ba1e/日本語_ΕΛΛΗΝΙΚΑ.jar 2016-12-01 11:50:22 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Beginning import of 日本語_ΕΛΛΗΝΙΚΑ 2016-12-01 11:50:22 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] mapred.jar is deprecated. Instead, use mapreduce.job.jar 2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Executing SQL statement: SELECT t.* FROM [日本語_ΕΛΛΗΝΙΚΑ] AS t WHERE 1=0 2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Executing SQL statement: SELECT t.* FROM [日本語_ΕΛΛΗΝΙΚΑ] AS t WHERE 1=0 2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Writing Avro schema file: /tmp/sqoop-adpqa/compile/9013a77a704c396d503bc9c56a95ba1e/日本語_ΕΛΛΗΝΙΚΑ.avsc 2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 2016-12-01 11:50:23 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] SQOOP_HOME is unset. May not be able to find all job dependencies. 2016-12-01 11:50:24 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] PriviledgedActionException as:adpqa (auth:KERBEROS) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error 2016-12-01 11:50:24 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error 2016-12-01 11:50:24 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] PriviledgedActionException as:adpqa (auth:KERBEROS) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error 2016-12-01 11:50:24 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Failing over to rm71 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Using read commited transaction isolation 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] BoundingValsQuery: SELECT MIN([C_MKTSGMT_CHINESE_市場]), MAX([C_MKTSGMT_CHINESE_市場]) FROM [日本語_ΕΛΛΗΝΙΚΑ] 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Generating splits for a textual index column. 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] If your database sorts in a case-insensitive order, this may result in a partial import or duplicate records. 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] You are strongly encouraged to choose an integral split column. 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Cleaning up the staging area /tmp/hadoop-yarn/staging/adpqa/.staging/job_1480054223215_0084 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] PriviledgedActionException as:adpqa (auth:PROXY) via adpqa (auth:KERBEROS) cause:java.io.IOException: 0.0015106201171875 is greater than 0.001205468783541194110069724588674860456249948725826470763422548770904541015625 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Encountered IOException running import job: java.io.IOException: 0.0015106201171875 is greater than 0.001205468783541194110069724588674860456249948725826470763422548770904541015625 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:204) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at java.security.AccessController.doPrivileged(Native Method) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at javax.security.auth.Subject.doAs(Subject.java:422) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at java.security.AccessController.doPrivileged(Native Method) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at javax.security.auth.Subject.doAs(Subject.java:422) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:203) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:176) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:273) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.manager.SQLServerManager.importTable(SQLServerManager.java:163) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.Sqoop.run(Sqoop.java:143) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at java.lang.reflect.Method.invoke(Method.java:497) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at com.infa.adapter.sqoop.client.Client.invoke(Client.java:86) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at com.infa.adapter.sqoop.client.Client.main(Client.java:40) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at java.lang.reflect.Method.invoke(Method.java:497) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.util.RunJar.run(RunJar.java:221) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Caused by: 0.0015106201171875 is greater than 0.001205468783541194110069724588674860456249948725826470763422548770904541015625 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:157) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:116) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:201) 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] ... 33 more 2016-12-01 11:50:25 <TASK_140221035255552-MAPPING> INFO: [APPSDK_Msg_1762] Thanks & Regards, Rohith -- This message was sent by Atlassian JIRA (v6.3.4#6332)