[
https://issues.apache.org/jira/browse/SQOOP-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16571394#comment-16571394
]
Fero Szabo commented on SQOOP-2949:
-----------------------------------
Hi [~gireeshp],
Great to hear!
We are trying to improve the test coverage of Sqoop so that we can ensure that
a new change won't break existing use cases. So the first step is to create a
few basic test cases for the change.
For this, you will need to install the docker images for the 3rd party tests if
you haven't done so, yet. Please see COMPILING.txt in the root directory of the
project for the details. (The relevant section starts at line 221: "Setting up
and executing third-party tests...")
*1. Creating tests*
As a start, I'd suggest modifying SQLServerSplitByTest.java. A test for SQL
server is a good start, though, since this is a fundamental change, a test for
the following RDBMS implementations would also be nice:
* Oracle
* MySQL
* PostgresSQL
The committers / reviewers might ask for these anyway, as Oracle is the most
popular database implementation used with Sqoop.
*2. Creating a review*
Please go to Review Board at [https://reviews.apache.org/account/login/] and
register if you haven't done so far. Then, create a patch by invoking _git diff
> SQOOP-2949-1.patch **_on the command line. Finally, create a review using the
_sqoop-trunk_ repository and your patch. Fill in the necessary fields, as for
example, in this reivew: [https://reviews.apache.org/r/65607/] (no need for a
description this long, nobody likes to read this much :) ).
Once this is done, we'll continue the technical discussion on ReviewBoard!
Please let me know if you've any questions here, or via email (you should be
able to see my email address under my name).
Best Regards,
Fero
> SQL Syntax error when split-by column is of character type and min or max
> value has single quote inside it
> ----------------------------------------------------------------------------------------------------------
>
> Key: SQOOP-2949
> URL: https://issues.apache.org/jira/browse/SQOOP-2949
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.6
> Environment: Sqoop 1.4.6
> Run on Hadoop 2.6.0
> On Ubuntu
> Reporter: Gireesh Puthumana
> Priority: Major
>
> Did a sqoop import from mysql table "emp", with split-by column "ename",
> which is a varchar(100) type.
> +Used below command:+
> sqoop import --connect jdbc:mysql://localhost/testdb --username root
> --password ***** --table emp --m 2 --target-dir /sqoopTest/5 --split-by ename;
> +Ename has following records:+
> | ename |
> | gireesh |
> | aavesh |
> | shiva' |
> | jamir |
> | balu |
> | santosh |
> | sameer |
> Min value is "aavesh" and max value is "shiva'" (please note the single quote
> inside max value).
> When run, it tried to execute below query in mapper 2 and failed:
> SELECT `ename`, `eid`, `deptid` FROM `emp` AS `emp` WHERE ( `ename` >=
> 'jd聯聭聪G耀' ) AND ( `ename` <= 'shiva'' )
> +Stack trace:+
> {quote}
> 2016-06-05 16:54:06,749 ERROR [main]
> org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception:
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error
> in your SQL syntax; check the manual that corresponds to your MySQL server
> version for the right syntax to use near ''shiva'' )' at line 1
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
> at com.mysql.jdbc.Util.getInstance(Util.java:387)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:942)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3966)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549)
> at
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)
> at
> com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1962)
> at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
