Hi All,
I have a question regarding the hive-jdbc library that is being included in the assembly JAR. Build command. mvn -U -X -Phadoop-2.6 -Phadoop-provided -Phive-provided -Pyarn -Phive-thriftserver -Psparkr -DskipTests install In the pom.xml file, the scope for hive JARs are set to 'compile', however, there is one entry https://github.com/apache/spark/blob/branch-1.6/pom.xml#L1414 [https://avatars1.githubusercontent.com/u/47359?v=3&s=400]<https://github.com/apache/spark/blob/branch-1.6/pom.xml#L1414> apache/spark<https://github.com/apache/spark/blob/branch-1.6/pom.xml#L1414> github.com spark - Mirror of Apache Spark that includes it again. The assembly JAR shows the following content with 'jar tf'. org/apache/hive/ org/apache/hive/jdbc/ org/apache/hive/jdbc/HiveDatabaseMetaData.class org/apache/hive/jdbc/ZooKeeperHiveClientHelper.class org/apache/hive/jdbc/ZooKeeperHiveClientHelper$DummyWatcher.class org/apache/hive/jdbc/HiveQueryResultSet$Builder.class org/apache/hive/jdbc/HiveResultSetMetaData.class org/apache/hive/jdbc/HivePreparedStatement.class org/apache/hive/jdbc/HiveStatement$1.class org/apache/hive/jdbc/JdbcUriParseException.class org/apache/hive/jdbc/HiveDataSource.class org/apache/hive/jdbc/HttpBasicAuthInterceptor.class org/apache/hive/jdbc/JdbcColumn.class org/apache/hive/jdbc/Utils$JdbcConnectionParams.class org/apache/hive/jdbc/HiveMetaDataResultSet.class org/apache/hive/jdbc/HiveDriver.class org/apache/hive/jdbc/JdbcTable.class org/apache/hive/jdbc/HiveBaseResultSet.class org/apache/hive/jdbc/HiveDatabaseMetaData$GetTablesComparator.class org/apache/hive/jdbc/HiveDatabaseMetaData$1.class org/apache/hive/jdbc/HiveStatement.class org/apache/hive/jdbc/ZooKeeperHiveClientException.class org/apache/hive/jdbc/HiveQueryResultSet$1.class org/apache/hive/jdbc/Utils.class org/apache/hive/jdbc/HiveConnection$1.class org/apache/hive/jdbc/JdbcColumn$1.class org/apache/hive/jdbc/HiveBaseResultSet$1.class org/apache/hive/jdbc/HttpKerberosRequestInterceptor.class org/apache/hive/jdbc/JdbcColumnAttributes.class org/apache/hive/jdbc/HiveCallableStatement.class org/apache/hive/jdbc/HiveDatabaseMetaData$GetColumnsComparator.class org/apache/hive/jdbc/ClosedOrCancelledStatementException.class org/apache/hive/jdbc/HiveQueryResultSet.class org/apache/hive/jdbc/HttpRequestInterceptorBase.class org/apache/hive/jdbc/HiveConnection.class org/apache/hive/service/ org/apache/hive/service/server/ org/apache/hive/service/server/HiveServerServerOptionsProcessor.class Would like to know why this is there and can we remove that? and link the hive-jdbc during runtime?
