I am reaching out once again to seek assistance and share a specific issue I've encountered while integrating Doris with Cloudera-flavored Hive. The error message I'm facing is as follows: detailMessage = get file split failed for table: baby1, err: java.lang.Exception: Hive 2.x versioned full-acid tables need to run major compaction.
This error message appears when I attempt to query a Hive table from Doris, and it seems to be related to Hive 2.x versioned full-ACID tables requiring a major compaction. I would like to ask the Doris community for guidance on how to handle this issue effectively. Specifically, I am interested in understanding the best practices and steps to follow when dealing with Cloudera-flavored Hive tables that require major compaction for Doris integration. If anyone in the community has successfully addressed this issue or can provide insights into how to configure and manage Cloudera-flavored Hive tables for integration with Doris, your expertise would be highly appreciated. Thank you for your time and support, and I look forward to receiving your valuable input. Joice On Wed, Oct 4, 2023 at 9:23 PM Joice Jacob <joicejacob1...@gmail.com> wrote: > Dear Doris Community, > I have an update on the issue I previously mentioned regarding the > integration of Hive on a CDP distribution with Doris 2.0.1.1. > > After further investigation, I have identified that the issue is related > to Hive managed tables being transactional by default. This appears to be > causing the error I encountered earlier. > > To address this issue, I am seeking guidance from the community on any > specific configurations or settings that need to be adjusted for Hive > transactional tables when using Hive as the catalog in Doris. Are there any > recommended configurations or best practices that I should follow to ensure > smooth integration and query execution? > > Any insights or recommendations from the Doris community would be greatly > appreciated. Your expertise and guidance will be instrumental in helping me > resolve this challenge. > > Thank you for your continued support, and I look forward to your valuable > input. > > Best regards, > Joice > > On Wed, Oct 4, 2023 at 6:54 PM Joice Jacob <joicejacob1...@gmail.com> > wrote: > >> Dear Doris Community, >> >> I am reaching out to the community to seek assistance with an integration >> issue I've encountered while trying to use Hive on a CDP distribution with >> Doris 2.0.1.1. >> >> Here are the details of my setup: >> >> Doris Version: 2.0.1.1 >> Hive Version: 3.1.3 >> Cluster Security: Kerberized >> >> I have successfully created a Hive catalog in Doris and have been able to >> set up the integration between Hive and Doris. However, when I attempt to >> query a Hive table using Doris, I encounter the following error: >> >> ERROR 1105 (HY000): errCode = 2, detailMessage = get file split failed >> for table: baby1, err: java.lang.Exception: Hive 2.x versioned full-acid >> tables need to run major compaction. >> >> I have already performed a major compaction as recommended, but I am >> still encountering the same error. >> >> I would greatly appreciate any insights, guidance, or solutions that the >> Doris community can offer to help me resolve this issue. If anyone has >> encountered a similar problem or has expertise in integrating Hive with >> Doris, your assistance would be invaluable. >> >> Thank you in advance for your time and support. I look forward to hearing >> from the community and working together to find a solution to this >> challenge. >> >> [image: baby_table.JPG] >> [image: doris_hive_catlog_result.JPG] >> >> [image: doris_fe_log.JPG] >> >> [image: hive_metastore_error.JPG] >> >> Thanks >> Joice >> >> >> >> >>
