[
https://issues.apache.org/jira/browse/IMPALA-14545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18037908#comment-18037908
]
ASF subversion and git services commented on IMPALA-14545:
----------------------------------------------------------
Commit 5f91838adaf55dd3a5818f5ad680254b292ebe1b in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=5f91838ad ]
IMPALA-14545: Don't use absolute hdfs paths for JDBC table driver.url
After IMPALA-13661 merged, S3PlannerTest.testDataSourceTables
has been failing with an error trying to fetch the JDBC driver
for functional.jdbc_decimal_tbl. This particular table's
definition uses a path like 'hdfs://localhost:20500/test-warehouse/...'
which explicitly depends on HDFS rather than relying
on the default filesystem. Changing this to use a path like
'/test-warehouse/...' without the HDFS dependency fixes the
S3PlannerTest. This changes create-ext-data-source-table.sql
to a template using WAREHOUSE_LOCATION_PREFIX and replaces that
variable before executing it. This is important for Ozone, as
Ozone uses a WAREHOUSE_LOCATION_PREFIX set to the Ozone volume.
Testing:
- Ran S3 and regular HDFS fe tests
Change-Id: I3f2c86fcc6c1dee75d7d9a9be04468cb197ae13c
Reviewed-on: http://gerrit.cloudera.org:8080/23658
Reviewed-by: Wenzhe Zhou <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> S3PlannerTest.testDataSourceTables fails with "Failed to copy JDBC driver"
> --------------------------------------------------------------------------
>
> Key: IMPALA-14545
> URL: https://issues.apache.org/jira/browse/IMPALA-14545
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure, Test
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Assignee: Pranav Yogi Lodha
> Priority: Blocker
>
> S3PlannerTest.testDataSourceTables is failing on S3 jobs with:
> {noformat}
> Query:
> select * from functional.jdbc_decimal_tbl where d3 > 123.456 and d5 < 10.0
> Error Stack:
> org.apache.impala.common.InternalException: Data source
> DataSource{name=impalajdbcdatasource, location=,
> className=org.apache.impala.extdatasource.jdbc.JdbcDataSource, apiVersion=V1}
> returned an error from prepare(): Failed to retrieve total number of records:
> Failed to copy JDBC driver from
> 'hdfs://localhost:20500/test-warehouse/data-sources/jdbc-drivers/postgresql-jdbc.jar'
> to local filesystem.{noformat}
> This seems to have started when IMPALA-13661 merged, but it is unclear why it
> started failing then.
> [https://github.com/apache/impala/commit/a77fec6391eaf5db7c52000c7446cc8582c770ac]
> I think the main problem for this is in the paths used in
> [https://github.com/apache/impala/blob/master/testdata/bin/create-ext-data-source-table.sql]
> . There are two main problems:
> 1. Some of the paths are absolute (e.g.
> [https://github.com/apache/impala/blob/master/testdata/bin/create-ext-data-source-table.sql#L90]
> )
> 2. The paths make no allowance for a FILESYSTEM_PREFIX or WAREHOUSE_PREFIX.
> Fixing the first one should fix S3. Since we don't run frontend tests on
> other non-HDFS filesystems, the second may not matter for this test.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
