On 12 Nov 2013, at 0:01, Sunita Arvind wrote:
Just in case this acts as a workaround for someone:
The issue is resolved if I eliminate the "where" clause in the query
(just
keep "where $CONDITIONS"). So 2 workarounds I can think of now are:
1. Create views in Oracle and query without the where clause in the
sqoop
import command
2. Import everything in the table (not feasible in most cases)
However, I still need to know how to get the exact stack trace.
regards
Sunita
On Mon, Nov 11, 2013 at 1:48 PM, Sunita Arvind
<sunitarv...@gmail.com>wrote:
Hello,
I am using sqoop to import data from oracle into hive. Below is my
SQL:
nohup sqoop import --connect "jdbc:oracle:thin:@(DESCRIPTION =
(ADDRESS =
(PROTOCOL = TCP)(HOST = xxxxxxx)(PORT = xxxx)) (CONNECT_DATA =
(SERVER =
DEDICATED) (SERVICE_NAME = CDWQ.tms.toyota.com) (FAILOVER_MODE=
(TYPE=select) (METHOD=basic))))" --username "xxxx" --password
"xxxx"
--split-by employeeid --query "SELECT e.employeeid,p.salary from
employee
e, payroll p
where e.employeeid =p.employeeid and $CONDITIONS"
--create-hive-table --hive-table "EMPLOYEE" --hive-import
--target-dir
"/user/hive/warehouse/employee" --direct --verbose
Note: This is production data hence I cannot share the log file or
actual
query. Sorry for that.
Similar query works for some tables and for this particular table,
there
is an exception as below:
java.io.IOException: SQLException in nextKeyValue
at
org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:266)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:484)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:76)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:85)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:673)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
This is usually the case when your PK (on which Sqoop will try to do the
split) isn't an integer.
my 2c.
David
