Hi Richa, Thats a work around. But how to handle the columns with INT type. Changing the type will be the last option for me.
Regards Biswa On Tue, Mar 1, 2016 at 9:31 AM, Richa Sharma <mailtorichasha...@gmail.com> wrote: > Hi Biswajit > > The answer is in the last line of the error message. Change the data type > of partition column to string in hive and try again. > > Hope it helps ! > > Richa > > 16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException running > export job: java.io.IOException: The table provided default.emp_details1 uses > unsupported partitioning key type for column salary : int. Only string > fields are allowed in partition columns in Catalog > > > On Tue, Mar 1, 2016 at 2:19 PM, Biswajit Nayak <biswa...@altiscale.com> > wrote: > >> Hi All, >> >> I am trying to do a SQOOP export from hive( integer type partition) to >> mysql through HCAT and it fails with the following error. >> >> Versions:- >> >> Hadoop :- 2.7.1 >> Hive :- 1.2.0 >> Sqoop :- 1.4.5 >> >> Table in Hive :- >> >> >> hive> use default; >> OK >> Time taken: 0.028 seconds >> hive> describe emp_details1; >> OK >> id int >> name string >> deg string >> dept string >> salary int >> >> # Partition Information >> # col_name data_type comment >> >> salary int >> Time taken: 0.125 seconds, Fetched: 10 row(s) >> hive> >> >> hive> select * from emp_details1; >> OK >> 1201 gopal 50000 >> 1202 manisha 50000 >> 1203 kalil 50000 >> 1204 prasanth 50000 >> 1205 kranthi 50000 >> 1206 satish 50000 >> Time taken: 0.195 seconds, Fetched: 6 row(s) >> hive> >> >> >> Conf added to Hive metastore site.xml >> >> >> [alti-test-01@hdpnightly271-ci-91-services ~]$ grep -A5 -B2 -i >> "hive.metastore.integral.jdo.pushdown" /etc/hive-metastore/hive-site.xml >> </property> >> <property> >> <name>hive.metastore.integral.jdo.pushdown</name> >> <value>TRUE</value> >> </property> >> >> </configuration> >> [alti-test-01@hdpnightly271-ci-91-services ~]$ >> >> >> The issue remains same >> >> >> [alti-test-01@hdpnightly271-ci-91-services ~]$ /opt/sqoop-1.4.5/bin/sqoop >> export --connect jdbc:mysql://localhost:3306/test --username hive --password >> ********* --table employee --hcatalog-database default --hcatalog-table >> emp_details1 >> Warning: /opt/sqoop-1.4.5/bin/../../hbase does not exist! HBase imports will >> fail. >> Please set $HBASE_HOME to the root of your HBase installation. >> Warning: /opt/sqoop-1.4.5/bin/../../accumulo does not exist! Accumulo >> imports will fail. >> Please set $ACCUMULO_HOME to the root of your Accumulo installation. >> Warning: /opt/sqoop-1.4.5/bin/../../zookeeper does not exist! Accumulo >> imports will fail. >> Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. >> 16/02/12 08:04:00 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5 >> 16/02/12 08:04:00 WARN tool.BaseSqoopTool: Setting your password on the >> command-line is insecure. Consider using -P instead. >> 16/02/12 08:04:00 INFO manager.MySQLManager: Preparing to use a MySQL >> streaming resultset. >> 16/02/12 08:04:00 INFO tool.CodeGenTool: Beginning code generation >> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: SELECT >> t.* FROM `employee` AS t LIMIT 1 >> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: SELECT >> t.* FROM `employee` AS t LIMIT 1 >> 16/02/12 08:04:01 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is >> /opt/hadoop >> Note: >> /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.java >> uses or overrides a deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> 16/02/12 08:04:07 INFO orm.CompilationManager: Writing jar file: >> /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.jar >> 16/02/12 08:04:07 INFO mapreduce.ExportJobBase: Beginning export of employee >> 16/02/12 08:04:08 INFO mapreduce.ExportJobBase: Configuring HCatalog for >> export job >> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Configuring HCatalog >> specific details for job >> 16/02/12 08:04:08 INFO manager.SqlManager: Executing SQL statement: SELECT >> t.* FROM `employee` AS t LIMIT 1 >> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column names >> projected : [id, name, deg, salary, dept] >> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column name - info >> map : >> id : [Type : 4,Precision : 11,Scale : 0] >> name : [Type : 12,Precision : 20,Scale : 0] >> deg : [Type : 12,Precision : 20,Scale : 0] >> salary : [Type : 4,Precision : 11,Scale : 0] >> dept : [Type : 12,Precision : 10,Scale : 0] >> >> 16/02/12 08:04:10 INFO hive.metastore: Trying to connect to metastore with >> URI thrift://hive-hdpnightly271-ci-91.test.altiscale.com:9083 >> 16/02/12 08:04:10 INFO hive.metastore: Connected to metastore. >> 16/02/12 08:04:11 INFO hcat.SqoopHCatUtilities: HCatalog full table schema >> fields = [id, name, deg, dept, salary] >> 16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException running >> export job: java.io.IOException: The table provided default.emp_details1 >> uses unsupported partitioning key type for column salary : int. Only >> string fields are allowed in partition columns in Catalog >> >> >> Stuck with this issue. Any one had conquered this before. >> >> Regards >> Biswa >> >> >
