[
https://issues.apache.org/jira/browse/HIVE-21263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Seung-Hyun Cheong updated HIVE-21263:
-------------------------------------
Description:
DruidStorageHandler should not disable the druid datasource of the hive
external table.
[DruidStorageHandler.java$L678|https://github.com/apache/hive/blob/master/druid-handler/src/java/org/apache/hadoop/hive/druid/DruidStorageHandler.java#L678]
{code:java}
if (deleteData && MetaStoreUtils.isExternalTablePurge(table)) {
LOG.info("Dropping with purge all the data for data source {}",
dataSourceName);
List<DataSegment>
dataSegmentList =
DruidStorageHandlerUtils.getDataSegmentList(getConnector(),
getDruidMetadataStorageTablesConfig(),
dataSourceName);
if (dataSegmentList.isEmpty()) {
LOG.info("Nothing to delete for data source {}", dataSourceName);
return;
}
for (DataSegment dataSegment : dataSegmentList) {
try {
deleteSegment(dataSegment);
} catch (SegmentLoadingException e) {
LOG.error(String.format("Error while deleting segment [%s]",
dataSegment.getIdentifier()), e);
}
}
}
if (DruidStorageHandlerUtils.disableDataSource(getConnector(),
getDruidMetadataStorageTablesConfig(),
dataSourceName)) {
LOG.info("Successfully dropped druid data source {}", dataSourceName);
}
{code}
If a druid datasource is disabled
# We can not query to a datasource.
# It takes too much time to enable again a datasource, if the datasource is
big.
The data of an external table should be usable, even after the external table
is dropped from hive.
So, DruidStorageHandler should not disable the druid datasource when a hive
external table is dropped.
was:
DruidStorageHandler should not disable the druid datasource of the hive
external table.
[DruidStorageHandler.java$L678|https://github.com/apache/hive/blob/master/druid-handler/src/java/org/apache/hadoop/hive/druid/DruidStorageHandler.java#L678]
{code:java}
if (deleteData && MetaStoreUtils.isExternalTablePurge(table)) {
LOG.info("Dropping with purge all the data for data source {}",
dataSourceName);
List<DataSegment>
dataSegmentList =
DruidStorageHandlerUtils.getDataSegmentList(getConnector(),
getDruidMetadataStorageTablesConfig(),
dataSourceName);
if (dataSegmentList.isEmpty()) {
LOG.info("Nothing to delete for data source {}", dataSourceName);
return;
}
for (DataSegment dataSegment : dataSegmentList) {
try {
deleteSegment(dataSegment);
} catch (SegmentLoadingException e) {
LOG.error(String.format("Error while deleting segment [%s]",
dataSegment.getIdentifier()), e);
}
}
}
if (DruidStorageHandlerUtils.disableDataSource(getConnector(),
getDruidMetadataStorageTablesConfig(),
dataSourceName)) {
LOG.info("Successfully dropped druid data source {}", dataSourceName);
}
{code}
If a druid datasource is disabled
# We can not query to a datasource.
# It takes too much time to enable again a datasource, if the datasource is
big.
The data of an external table should be usable, even after the external table
is dropped from hive.
So, DruidStorageHandler should not disable the druid datasource when a hive
external table is dropped.
> DruidStorageHandler should not disable the druid datasource when a hive
> external table is dropped
> -------------------------------------------------------------------------------------------------
>
> Key: HIVE-21263
> URL: https://issues.apache.org/jira/browse/HIVE-21263
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
> Affects Versions: 3.1.0
> Reporter: Seung-Hyun Cheong
> Assignee: Seung-Hyun Cheong
> Priority: Minor
>
> DruidStorageHandler should not disable the druid datasource of the hive
> external table.
>
> [DruidStorageHandler.java$L678|https://github.com/apache/hive/blob/master/druid-handler/src/java/org/apache/hadoop/hive/druid/DruidStorageHandler.java#L678]
> {code:java}
> if (deleteData && MetaStoreUtils.isExternalTablePurge(table)) {
> LOG.info("Dropping with purge all the data for data source {}",
> dataSourceName);
> List<DataSegment>
> dataSegmentList =
> DruidStorageHandlerUtils.getDataSegmentList(getConnector(),
> getDruidMetadataStorageTablesConfig(),
> dataSourceName);
> if (dataSegmentList.isEmpty()) {
> LOG.info("Nothing to delete for data source {}", dataSourceName);
> return;
> }
> for (DataSegment dataSegment : dataSegmentList) {
> try {
> deleteSegment(dataSegment);
> } catch (SegmentLoadingException e) {
> LOG.error(String.format("Error while deleting segment [%s]",
> dataSegment.getIdentifier()), e);
> }
> }
> }
> if (DruidStorageHandlerUtils.disableDataSource(getConnector(),
> getDruidMetadataStorageTablesConfig(),
> dataSourceName)) {
> LOG.info("Successfully dropped druid data source {}", dataSourceName);
> }
> {code}
>
> If a druid datasource is disabled
> # We can not query to a datasource.
> # It takes too much time to enable again a datasource, if the datasource is
> big.
>
> The data of an external table should be usable, even after the external table
> is dropped from hive.
>
> So, DruidStorageHandler should not disable the druid datasource when a hive
> external table is dropped.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
