[
https://issues.apache.org/jira/browse/IMPALA-13931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17950615#comment-17950615
]
ASF subversion and git services commented on IMPALA-13931:
----------------------------------------------------------
Commit afa329fd8961f766c48bb81859b47b078d5e2b55 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=afa329fd8 ]
IMPALA-13931: TestIcebergRestCatalog.test_rest_catalog_basic failed at setup
There were several issues with test_rest_catalog_basic which made it
fail in environments that used Ozone or S3.
Missing dependency of Ozone and S3 classes:
* This is resolved in iceberg-rest-catalog-test/pom.xml by adding
a dependency to impala-executor-deps
Hadoop configuration was initialized properly:
* run-iceberg-rest-server.sh used Maven to run Iceberg REST Catalog in
which case Maven is in charge of setting the CLASSPATH but the
core-site/ozone-site/etc. config files were not on it, so the
REST Catalog used a default Hadoop configuration that wasn't good
for our environment.
* To overcome the CLASSPATH problem now we create a runnable JAR in
iceberg-rest-catalog-test/pom.xml and also generate the proper
CLASSPATH during compilation.
* run-iceberg-rest-server.sh now uses java -cp to run the REST
Catalog
S3 builds threw NoSuchMethodException for the "create" method of
ApacheHttpClientConfigurations:
* The Iceberg library dynamically load its http client builders
to workaround an error, see details in
https://github.com/apache/iceberg/issues/6715
* So the Iceberg lib dynamically wants to load the "create" method
of its own ApacheHttpClientConfigurations class but it fails
with NoSuchMethodException.
* The critical code is invoked from Impala's IcebergMetadataScanner's
ScanMetadataTable() method which happens to be invoked through
JNI from the C++ backend.
* The context class loader of such threads are NULL, which means
Java will use the bootstrap class loader to load classes and methods,
but that doesn't have the proper resources on its classpath.
* To overcome this issue we set the context class loader for the thread
to the class loader that originally loaded the IcebergMetadataScanner
class.
Change-Id: I9dc0e30aeaff0b8de41426ba38506383b4af472c
Reviewed-on: http://gerrit.cloudera.org:8080/22818
Reviewed-by: Jason Fehr <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Riza Suminto <[email protected]>
> TestIcebergRestCatalog.test_rest_catalog_basic failed at setup
> --------------------------------------------------------------
>
> Key: IMPALA-13931
> URL: https://issues.apache.org/jira/browse/IMPALA-13931
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Yida Wu
> Assignee: Zoltán Borók-Nagy
> Priority: Blocker
> Labels: broken-build, test-failure
>
> Observed a test failure in the builds with TestIcebergRestCatalog's
> test_rest_catalog_basic, likely related to IMPALA-13586, which introduced
> this test case.
> {code:java}
> custom_cluster.test_iceberg_rest_catalog.TestIcebergRestCatalog.test_rest_catalog_basic[protocol:
> beeswax | table_format: text/none | exec_option: {'test_replan': 1,
> 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
> 'disable_codegen': False, 'abort_on_error': 1,
> 'exec_single_node_rows_threshold': 0}]
> {code}
> Error Message
> {code:java}
> test setup failure
> {code}
> Stacktrace
> {code:java}
> custom_cluster/test_iceberg_rest_catalog.py:55: in setup_method
> raise e
> E Exception: Webserver did not become available within 300 seconds.
> {code}
> Standard Output
> {code:java}
> ...
> IMPALA_VERSION = 5.0.0-SNAPSHOT
> ...
> [INFO]
> [INFO] ---------< org.apache.impala:impala-iceberg-rest-catalog-test
> >---------
> [INFO] Building Iceberg REST Catalog Test 5.0.0-SNAPSHOT
> [INFO] from pom.xml
> [INFO] --------------------------------[ jar
> ]---------------------------------
> ...
> [INFO] Downloading from cloudera-mirrors:
> http://maven.jenkins.cloudera.com:8081/artifactory/cloudera-mirrors/org/apache/iceberg/iceberg-core/1.5.2.2025.0.20.0-81/iceberg-core-1.5.2.2025.0.20.0-81-tests.jar
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 3.652 s
> [INFO] Finished at: 2025-04-03T14:54:46-07:00
> [INFO]
> ------------------------------------------------------------------------
> [WARNING]
> [WARNING] Plugin validation issues were detected in 1 plugin(s)
> [WARNING]
> [WARNING] * org.apache.maven.plugins:maven-site-plugin:3.12.1
> [WARNING]
> [WARNING] For more or less details, use 'maven.plugin.validation' property
> with one of the values (case insensitive): [BRIEF, DEFAULT, VERBOSE]
> [WARNING]
> [ERROR] Failed to execute goal on project impala-iceberg-rest-catalog-test:
> Could not resolve dependencies for project
> org.apache.impala:impala-iceberg-rest-catalog-test:jar:5.0.0-SNAPSHOT: The
> following artifacts could not be resolved:
> org.apache.hadoop:hadoop-common:jar:3.1.1.7.2.18.400-40 (present, but
> unavailable), org.apache.hadoop:hadoop-hdfs-client:jar:3.1.1.7.2.18.400-40
> (present, but unavailable),
> org.apache.iceberg:iceberg-api:jar:1.5.2.2025.0.20.0-81 (present, but
> unavailable), org.apache.iceberg:iceberg-core:jar:1.5.2.2025.0.20.0-81
> (present, but unavailable),
> org.apache.iceberg:iceberg-core:jar:tests:1.5.2.2025.0.20.0-81 (present, but
> unavailable): Could not find artifact
> org.apache.hadoop:hadoop-common:jar:3.1.1.7.2.18.400-40 in cloudera-mirrors
> (http://maven.jenkins.cloudera.com:8081/artifactory/cloudera-mirrors) ->
> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
