[
https://issues.apache.org/jira/browse/SQOOP-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962916#comment-16962916
]
Boglarka Egyed commented on SQOOP-3453:
---------------------------------------
[~saiyam1712] Sqoop import into S3 is not supported with this version, it has
been introduced only by SQOOP-3345 but has not been included into any official
release yet. It works in trunk version though.
Furthermore, Parquet import into S3 will require some more options to set as
traditionally Sqoop used Kite SDK to read/write Parquet that has been changed
in SQOOP-3313 because it has many limitations. This change has also not been
released yet but can be found in trunk.
> Kite sdk issue with sqoop version 1.4.7
> ---------------------------------------
>
> Key: SQOOP-3453
> URL: https://issues.apache.org/jira/browse/SQOOP-3453
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/oracle
> Affects Versions: 1.4.7
> Reporter: Saiyam Agarwal
> Priority: Major
>
>
> Trying to run sqoop import command with sqoop version 1.4.7 and kite sdk
> version 1.1.0.
> {code:java}
> /usr/lib/sqoop/bin/sqoop import -Dmapreduce.map.memory.mb=2048
> -Dmapred.fairscheduler.pool=sqoop-coop -Doraoop.timestamp.string=false
> --connect jdbc:oracle:thin:@abc.net:1521/coopdvm --username abc
> --password-file s3://abc/user/app.ab.core_etl/credentials/coopdvm_coop
> --target-dir 's3://abc/dev/temp/TEST_MEIS/T1/master/timestamp=20191017103331'
> --direct --as-parquetfile -z --map-column-java ID=Integer --query "select ID
> from TEST_MEIS.T1 where \$CONDITIONS" -m 1 --verbose{code}
> Failing with below issue:
> {code:java}
> App > 19/10/23 05:58:04 DEBUG util.ClassLoaderStack: Restoring classloader:
> sun.misc.Launcher$AppClassLoader@642c39d2
> App > 19/10/23 05:58:04 DEBUG manager.OracleManager$ConnCache: Caching
> released connection for jdbc:oracle:thin:@abc:1521/coopdvm/coop
> App > 19/10/23 05:58:04 ERROR sqoop.Sqoop: Got exception running Sqoop:
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://abc/dev/temp/TEST_MEIS/T1/master/timestamp=20191017103331
> App > Check that JARs for s3 datasets are on the classpath
> App > org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern:
> dataset:s3://abc/dev/temp/TEST_MEIS/T1/master/timestamp=20191017103331
> App > Check that JARs for s3 datasets are on the classpath
> App > at
> org.kitesdk.data.spi.Registration.lookupDatasetUri(Registration.java:128)
> App > at org.kitesdk.data.Datasets.exists(Datasets.java:624)
> App > at org.kitesdk.data.Datasets.exists(Datasets.java:646)
> App > at
> org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:118)
> {code}
> Can anyone help in debugging or suggest any workaround?
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
